+1 on starting with the Wiki. I really like the name "Engine room". Can we name the section in the Wiki like that? In general, if we think that a post or a series of posts would be a good fit for the main blog, it would be pretty straightforward to promote a post from the Engine room to the main blog (including further edits, focus on language, etc.)
– Ufuk On Tue, Mar 3, 2020 at 5:58 PM Rong Rong <walter...@gmail.com> wrote: > Big +1 on this. Some of these topics are not only for contributors, but > would also be super useful for advance users. > One topic I can think of in addition is: Security/Kerberos. > > Echo on Both Seth's idea, we could have both wiki and PR submission: > As Robert mentioned - wiki submission would make the experience more > frictionless. > I was having concerns splitting the blog posts in two places, but I also > think adding the banner/blog-series of "Flink Engine Room" would help > readers distinct between the two. > > -- > Rong > > On Tue, Mar 3, 2020 at 8:39 AM Dian Fu <dian0511...@gmail.com> wrote: > > > Big +1 on this idea. It will benefit both the developers and users a lot. > > > > Regarding to the place to hold these blogs, my preference is 3) as I > > notice that there are already a few high quality blogs on flink > web-site[1] > > and I guess that may be a good place to start with. We just need to > figure > > out a way to let contributors clearly mark the audience of their articles > > and also help users to easily determine whether the content is what they > > want. > > > > Regards, > > Dian > > > > [1] https://flink.apache.org/blog/ <https://flink.apache.org/blog/> > > > 在 2020年3月3日，下午11:14，Yadong Xie <vthink...@gmail.com> 写道： > > > > > > Hi all > > > > > > maybe we can use markdown & GitHub to make the submission easy to > review > > > I have set up a similar blog for Flink-china blog before(deprecated), > > glad > > > to offer help if needed > > > > > > here is the link: https://github.com/flink-china/doc > > > > > > Seth Wiesman <sjwies...@gmail.com> 于2020年3月3日周二 下午10:51写道： > > > > > >> For lack of a better way to put this, I think the location depends on > > the > > >> level of effort you want to put into writing these articles. > > >> > > >> If they are informal design documents then I think the wiki is the way > > to > > >> go. > > >> > > >> If you want to have them be more polished then the existing blog. This > > >> means going through a PR on the flink website, thinking about > language, > > >> etc. If we go this route we can distinguish them with a series title > > like > > >> "Flink Engine Room" and a disclaimer at the top. > > >> > > >> "Flink Engine Room: Plugins" > > >> > > >> "Flink Engine Room is a series of blog posts covering ongoing > > development > > >> on Apache Flink internals, why decisions were made, and how they will > > >> impact future development. The information described in this post is > not > > >> required to successfully write and deploy Flink applications in > > >> production." > > >> > > >> Seth > > >> > > >> > > >> On Tue, Mar 3, 2020 at 8:29 AM Arvid Heise <ar...@ververica.com> > wrote: > > >> > > >>> I think there is enough to positive to start setting it up. That begs > > the > > >>> question: in which format. > > >>> > > >>> Following possibilities exist: > > >>> 1) Use wiki as Robert pointed out. > > >>> 2) Add new blog. > > >>> 3) Use existing blog and separate by tags #user, #expert, #dev (can > be > > >>> mixed). Start page could filter on #user by default. > > >>> 4) ??? > > >>> > > >>> I'm assuming only few have a strong opinion, so I'd be happy if you'd > > >> just > > >>> drop your numbers in order of highest to lowest preference. > > >>> > > >>> On Tue, Mar 3, 2020 at 2:48 PM Piotr Nowojski <pi...@ververica.com> > > >> wrote: > > >>> > > >>>> +1 for the idea :) And fully agree to clearly separate them. > > >>>> > > >>>> I think the original idea was writing about some recent changes in > the > > >>>> Flink’s code base, that could affect other Flink developers > > >>>> (contributors/committers). Like for example some new ideas/future > > >>>> directions that we want to follow. Especially if they are work in > > >>> progress > > >>>> and there is lots of old code not adhering to those new ideas. In > some > > >>>> later responses, it seemed like people are more thinking about > > >> presenting > > >>>> some more advanced features, like a deep tech dive for power users. > > >>>> > > >>>> I’m not opposing the deep tech dives, but I just wanted to note that > > >> is a > > >>>> different target audience. I think the dev blogs could cover both of > > >>> them. > > >>>> At least initially. Later on we can decide to put more emphasis on > > >> power > > >>>> users or Flink devs, or split them, or whatever. > > >>>> > > >>>> Piotrek > > >>>> > > >>>>> On 3 Mar 2020, at 12:37, Jingsong Li <jingsongl...@gmail.com> > wrote: > > >>>>> > > >>>>> +1 for this proposal. I have a lot of desired topics in table and > > >>> batch. > > >>>>> > > >>>>> I also second Seth and Stephan 's comment separate this in a clear > > >> way. > > >>>>> Have concerns that maybe easy to confuse new users. > > >>>>> If I am a beginner and find a bunch of deep documents, I need to > > >>> further > > >>>>> distinguish which is effective and which is invalid for me, which > may > > >>>> cause > > >>>>> me a lot of trouble. > > >>>>> > > >>>>> Best, > > >>>>> Jingsong Lee > > >>>>> > > >>>>> On Tue, Mar 3, 2020 at 6:36 PM Flavio Pompermaier < > > >>> pomperma...@okkam.it> > > >>>>> wrote: > > >>>>> > > >>>>>> Big +1 from my side. I'd be very interested in what Jeff proposed, > > >> in > > >>>>>> particular everything related to client part (job submission, > > >> workflow > > >>>>>> management, callbacks on submission/success/failure, etc). > > >>>>>> Something I can't find anywhere is also how to query Flink > > >>>> states..would it > > >>>>>> be possible to have something like the Presto UI [1]? Does Flink > > >>>> implement > > >>>>>> some sort of query queuing? I heard about a query proxy server > but I > > >>>> don't > > >>>>>> know if there's a will to push in that direction. > > >>>>>> For Stateful Functions it would be nice to deeply compare the taxi > > >>>> driver > > >>>>>> solution with a more common implementation (i.e. using a database > to > > >>>>>> persist the legal data..is it safe to keep them as a Flink > state?). > > >>>>>> [1] > > >>>> > https://www.tutorialspoint.com/apache_presto/images/web_interface.jpg > > >>>>>> > > >>>>>> Best, > > >>>>>> Flavio > > >>>>>> > > >>>>>> On Tue, Mar 3, 2020 at 10:47 AM Jeff Zhang <zjf...@gmail.com> > > >> wrote: > > >>>>>> > > >>>>>>> +1 for this proposal. I am preparing some articles for how to > use > > >>>> Flink > > >>>>>> on > > >>>>>>> Zeppelin, although it is not closely related with this topic, but > > >>>> should > > >>>>>> be > > >>>>>>> helpful for users to get started with Flink. > > >>>>>>> > > >>>>>>> Till Rohrmann <trohrm...@apache.org> 于2020年3月3日周二 下午5:39写道： > > >>>>>>> > > >>>>>>>> I like the idea. +1 from my side. > > >>>>>>>> > > >>>>>>>> Potential topics: > > >>>>>>>> - Scheduling > > >>>>>>>> - Cluster partitions > > >>>>>>>> - Memory configuration > > >>>>>>>> - Recovery > > >>>>>>>> > > >>>>>>>> Cheers, > > >>>>>>>> Till > > >>>>>>>> > > >>>>>>>> On Tue, Mar 3, 2020 at 3:56 AM Xintong Song < > > >> tonysong...@gmail.com> > > >>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> Big +1. Thanks for the idea, Arvid. > > >>>>>>>>> > > >>>>>>>>> I'd be excited to read such blogs. > > >>>>>>>>> > > >>>>>>>>> And we would also be happy to contribute some contents on the > > >>> newest > > >>>>>>>>> efforts from our team. > > >>>>>>>>> Potential topics: > > >>>>>>>>> - Memory configuration > > >>>>>>>>> - Active Kubernetes integration > > >>>>>>>>> - GPU support > > >>>>>>>>> - Pluggable (dynamic) slot allocation > > >>>>>>>>> > > >>>>>>>>> Thank you~ > > >>>>>>>>> > > >>>>>>>>> Xintong Song > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> On Tue, Mar 3, 2020 at 9:59 AM Benchao Li <libenc...@gmail.com > > > > >>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> +1 for this proposal. As a contributor, it would be very > helpful > > >>> to > > >>>>>>>> have > > >>>>>>>>>> such blogs for us to understand status and future of Flink. > > >>>>>>>>>> > > >>>>>>>>>> Robert Metzger <rmetz...@apache.org> 于2020年3月3日周二 上午6:00写道： > > >>>>>>>>>> > > >>>>>>>>>>> I would be excited to read such a blog (can I request topics? > > >> :) > > >>>>>> ) > > >>>>>>>>>>> > > >>>>>>>>>>> We could start very low key by using our wiki's blog feature: > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/pages/viewrecentblogposts.action?key=FLINK > > >>>>>>>>>>> > > >>>>>>>>>>> On Mon, Mar 2, 2020 at 8:26 PM Stephan Ewen < > se...@apache.org> > > >>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> Great idea, but I also second Seth's comment to separate > this > > >>>>>> in > > >>>>>>> a > > >>>>>>>>>> clear > > >>>>>>>>>>>> way. It's easy to confuse new / potential users. > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Mon, Mar 2, 2020 at 8:15 PM Seth Wiesman < > > >>>>>> sjwies...@gmail.com > > >>>>>>>> > > >>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> +1 on the idea. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> My only request would be they are clearly marked as being > > >>>>>> about > > >>>>>>>>>>>> internals / > > >>>>>>>>>>>>> for advanced users to not give typical users the wrong > > >>>>>>> impression > > >>>>>>>>>> about > > >>>>>>>>>>>> how > > >>>>>>>>>>>>> much they need to understand to use Flink. Nico's network > > >>>>>> stack > > >>>>>>>>> blog > > >>>>>>>>>>> post > > >>>>>>>>>>>>> does this well[1]. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Seth > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> [1] > > >>>>>>> https://flink.apache.org/2019/06/05/flink-network-stack.html > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On Mon, Mar 2, 2020 at 10:39 AM Ufuk Celebi < > u...@apache.org> > > >>>>>>>>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> I'd be happy to read such a blog. Big +1 as a potential > > >>>>>>> reader. > > >>>>>>>>> ;-) > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> – Ufuk > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Mon, Mar 2, 2020 at 11:53 AM Arvid Heise < > > >>>>>>>> ar...@ververica.com > > >>>>>>>>>> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Dear devs, > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> development speed of Flink has steadily increased. Lots > > >>>>>> of > > >>>>>>>> new > > >>>>>>>>>>>> concepts > > >>>>>>>>>>>>>> are > > >>>>>>>>>>>>>>> introduced and technical debt removed. However, it's hard > > >>>>>>> to > > >>>>>>>>> keep > > >>>>>>>>>>>> track > > >>>>>>>>>>>>>> of > > >>>>>>>>>>>>>>> these things if you are not directly involved. Especially > > >>>>>>> for > > >>>>>>>>> new > > >>>>>>>>>>>>>>> contributors, it's often not easy to know what the best > > >>>>>>>>> practices > > >>>>>>>>>>> are > > >>>>>>>>>>>>> or > > >>>>>>>>>>>>>> if > > >>>>>>>>>>>>>>> there are related work streams going on. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> In the runtime team, we had the idea to set up a dev blog > > >>>>>>>> where > > >>>>>>>>>> we > > >>>>>>>>>>>>> could > > >>>>>>>>>>>>>>> introduce newest developments. The scope should be expert > > >>>>>>>> users > > >>>>>>>>>>> that > > >>>>>>>>>>>>>>> contribute to the project. Of course, some articles may > > >>>>>>> have > > >>>>>>>> a > > >>>>>>>>>>>> broader > > >>>>>>>>>>>>>>> scope and even be linked from release notes. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Examples from our team to give a more specific idea: > > >>>>>>>>>>>>>>> * Deprecated checkpoint lock and mailbox model > > >>>>>>>>>>>>>>> * Revised interface for two phase commit sinks and new > > >>>>>> JDBC > > >>>>>>>>> sink > > >>>>>>>>>>>>>>> * N-ary input operators > > >>>>>>>>>>>>>>> * Unaligned checkpoints > > >>>>>>>>>>>>>>> * Operator factories > > >>>>>>>>>>>>>>> * Plugins > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> These articles would be less formal than a FLIP (but > > >>>>>> could > > >>>>>>>> link > > >>>>>>>>>> to > > >>>>>>>>>>>> them > > >>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>> avoid redundancy) and focus more on how other developers > > >>>>>>> are > > >>>>>>>>>>> actually > > >>>>>>>>>>>>>>> impacted by the changes. It can also be used to share > > >>>>>>>>> experiences > > >>>>>>>>>>>>> during > > >>>>>>>>>>>>>>> the implementation. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> We would like to ask the other teams if they see a > > >>>>>> benefit > > >>>>>>> of > > >>>>>>>>>> such > > >>>>>>>>>>> a > > >>>>>>>>>>>>> blog > > >>>>>>>>>>>>>>> and would like to contribute. Bonus points if you could > > >>>>>>>> provide > > >>>>>>>>>>>>> examples > > >>>>>>>>>>>>>> of > > >>>>>>>>>>>>>>> your topics. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Best, > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Arvid > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> -- > > >>>>>>>>>> > > >>>>>>>>>> Benchao Li > > >>>>>>>>>> School of Electronics Engineering and Computer Science, Peking > > >>>>>>>> University > > >>>>>>>>>> Tel:+86-15650713730 > > >>>>>>>>>> Email: libenc...@gmail.com; libenc...@pku.edu.cn > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> Best Regards > > >>>>>>> > > >>>>>>> Jeff Zhang > > >>>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> Best, Jingsong Lee > > >>>> > > >>>> > > >>> > > >> > > > > >
