Big +1 on adding a dev blog and starting with wiki. And +1 to promote the fully polished articles to blog web with a formal process.
The latter one also brings up another good-to-have improvement that adding categories and navigation in our blog so people could easily find different topics like release-announcement/events/tech-articles, etc. but I think we'd better open another thread to keep this one on track (smile). I'd also like to add one potential topic around in-production practice of using RocksDB state backend (which seems to be a popular topic in ML discussions), such as how to enable and monitor RocksDB metrics and do debugging/perf-tuning with the metrics/logs, and introduce internals/details around the RocksDB memory management mechanism. Best Regards, Yu On Wed, 4 Mar 2020 at 11:07, Xintong Song <tonysong...@gmail.com> wrote: > I also like Ufuk's idea. > > The wiki allows people to post on their works in a quick and easier way. > For me and probably many other Chinese folks, writing and polishing a > formal article in English usually takes a long time, of which a significant > portion is spent on polishing the language. If the blog does not require > such formal and high quality languages, I believe it will make things a lot > easier and encourage more people to share their ideas. Besides, it also > avoids putting more review workloads on committers. > > Regarding promoting wiki post to the main blog, I think the wiki feedbacks > (comment, likes, etc.) could be a great input. We can also contact the > original author before promoting posts to the main blog to refine the > article (responding to the wiki comments, polishing languages, adding > latest updates, etc.). > > Thank you~ > > Xintong Song > > > > On Wed, Mar 4, 2020 at 10:25 AM Jark Wu <imj...@gmail.com> wrote: > > > +1 for this. > > > > Regarding to the place to hold blogs. Personally, I prefer to use > existing > > blog and separate by tags/categories and title names. > > Because, the dev blogs are very good learning materials. I believe many > > users will be interested in these posts. It's just like > > "Technology Deep Dive" talks in Flink Forward which attracts many > > audiences. Putting them with main blog together can help > > to give the dev blogs more exposure. > > > > But I also share Robert's concern. So I'm in favor of Ufuk's idea: > starting > > with Wiki, and moving good posts to the main blog gradually. > > We should also improve our current blog web to support tags/categories. > > Maybe @vthink...@gmail.com <vthink...@gmail.com> Yadong can help on > this. > > > > Best, > > Jark > > > > > > On Wed, 4 Mar 2020 at 05:03, Ufuk Celebi <u...@apache.org> wrote: > > > > > +1 on starting with the Wiki. I really like the name "Engine room". Can > > we > > > name the section in the Wiki like that? In general, if we think that a > > post > > > or a series of posts would be a good fit for the main blog, it would be > > > pretty straightforward to promote a post from the Engine room to the > main > > > blog (including further edits, focus on language, etc.) > > > > > > – Ufuk > > > > > > > > > On Tue, Mar 3, 2020 at 5:58 PM Rong Rong <walter...@gmail.com> wrote: > > > > > > > Big +1 on this. Some of these topics are not only for contributors, > but > > > > would also be super useful for advance users. > > > > One topic I can think of in addition is: Security/Kerberos. > > > > > > > > Echo on Both Seth's idea, we could have both wiki and PR submission: > > > > As Robert mentioned - wiki submission would make the experience more > > > > frictionless. > > > > I was having concerns splitting the blog posts in two places, but I > > also > > > > think adding the banner/blog-series of "Flink Engine Room" would help > > > > readers distinct between the two. > > > > > > > > -- > > > > Rong > > > > > > > > On Tue, Mar 3, 2020 at 8:39 AM Dian Fu <dian0511...@gmail.com> > wrote: > > > > > > > > > Big +1 on this idea. It will benefit both the developers and users > a > > > lot. > > > > > > > > > > Regarding to the place to hold these blogs, my preference is 3) as > I > > > > > notice that there are already a few high quality blogs on flink > > > > web-site[1] > > > > > and I guess that may be a good place to start with. We just need to > > > > figure > > > > > out a way to let contributors clearly mark the audience of their > > > articles > > > > > and also help users to easily determine whether the content is what > > > they > > > > > want. > > > > > > > > > > Regards, > > > > > Dian > > > > > > > > > > [1] https://flink.apache.org/blog/ <https://flink.apache.org/blog/ > > > > > > > > 在 2020年3月3日，下午11:14，Yadong Xie <vthink...@gmail.com> 写道： > > > > > > > > > > > > Hi all > > > > > > > > > > > > maybe we can use markdown & GitHub to make the submission easy to > > > > review > > > > > > I have set up a similar blog for Flink-china blog > > before(deprecated), > > > > > glad > > > > > > to offer help if needed > > > > > > > > > > > > here is the link: https://github.com/flink-china/doc > > > > > > > > > > > > Seth Wiesman <sjwies...@gmail.com> 于2020年3月3日周二 下午10:51写道： > > > > > > > > > > > >> For lack of a better way to put this, I think the location > depends > > > on > > > > > the > > > > > >> level of effort you want to put into writing these articles. > > > > > >> > > > > > >> If they are informal design documents then I think the wiki is > the > > > way > > > > > to > > > > > >> go. > > > > > >> > > > > > >> If you want to have them be more polished then the existing > blog. > > > This > > > > > >> means going through a PR on the flink website, thinking about > > > > language, > > > > > >> etc. If we go this route we can distinguish them with a series > > title > > > > > like > > > > > >> "Flink Engine Room" and a disclaimer at the top. > > > > > >> > > > > > >> "Flink Engine Room: Plugins" > > > > > >> > > > > > >> "Flink Engine Room is a series of blog posts covering ongoing > > > > > development > > > > > >> on Apache Flink internals, why decisions were made, and how they > > > will > > > > > >> impact future development. The information described in this > post > > is > > > > not > > > > > >> required to successfully write and deploy Flink applications in > > > > > >> production." > > > > > >> > > > > > >> Seth > > > > > >> > > > > > >> > > > > > >> On Tue, Mar 3, 2020 at 8:29 AM Arvid Heise <ar...@ververica.com > > > > > > wrote: > > > > > >> > > > > > >>> I think there is enough to positive to start setting it up. > That > > > begs > > > > > the > > > > > >>> question: in which format. > > > > > >>> > > > > > >>> Following possibilities exist: > > > > > >>> 1) Use wiki as Robert pointed out. > > > > > >>> 2) Add new blog. > > > > > >>> 3) Use existing blog and separate by tags #user, #expert, #dev > > (can > > > > be > > > > > >>> mixed). Start page could filter on #user by default. > > > > > >>> 4) ??? > > > > > >>> > > > > > >>> I'm assuming only few have a strong opinion, so I'd be happy if > > > you'd > > > > > >> just > > > > > >>> drop your numbers in order of highest to lowest preference. > > > > > >>> > > > > > >>> On Tue, Mar 3, 2020 at 2:48 PM Piotr Nowojski < > > pi...@ververica.com > > > > > > > > > >> wrote: > > > > > >>> > > > > > >>>> +1 for the idea :) And fully agree to clearly separate them. > > > > > >>>> > > > > > >>>> I think the original idea was writing about some recent > changes > > in > > > > the > > > > > >>>> Flink’s code base, that could affect other Flink developers > > > > > >>>> (contributors/committers). Like for example some new > > ideas/future > > > > > >>>> directions that we want to follow. Especially if they are work > > in > > > > > >>> progress > > > > > >>>> and there is lots of old code not adhering to those new ideas. > > In > > > > some > > > > > >>>> later responses, it seemed like people are more thinking about > > > > > >> presenting > > > > > >>>> some more advanced features, like a deep tech dive for power > > > users. > > > > > >>>> > > > > > >>>> I’m not opposing the deep tech dives, but I just wanted to > note > > > that > > > > > >> is a > > > > > >>>> different target audience. I think the dev blogs could cover > > both > > > of > > > > > >>> them. > > > > > >>>> At least initially. Later on we can decide to put more > emphasis > > on > > > > > >> power > > > > > >>>> users or Flink devs, or split them, or whatever. > > > > > >>>> > > > > > >>>> Piotrek > > > > > >>>> > > > > > >>>>> On 3 Mar 2020, at 12:37, Jingsong Li <jingsongl...@gmail.com > > > > > > wrote: > > > > > >>>>> > > > > > >>>>> +1 for this proposal. I have a lot of desired topics in table > > and > > > > > >>> batch. > > > > > >>>>> > > > > > >>>>> I also second Seth and Stephan 's comment separate this in a > > > clear > > > > > >> way. > > > > > >>>>> Have concerns that maybe easy to confuse new users. > > > > > >>>>> If I am a beginner and find a bunch of deep documents, I need > > to > > > > > >>> further > > > > > >>>>> distinguish which is effective and which is invalid for me, > > which > > > > may > > > > > >>>> cause > > > > > >>>>> me a lot of trouble. > > > > > >>>>> > > > > > >>>>> Best, > > > > > >>>>> Jingsong Lee > > > > > >>>>> > > > > > >>>>> On Tue, Mar 3, 2020 at 6:36 PM Flavio Pompermaier < > > > > > >>> pomperma...@okkam.it> > > > > > >>>>> wrote: > > > > > >>>>> > > > > > >>>>>> Big +1 from my side. I'd be very interested in what Jeff > > > proposed, > > > > > >> in > > > > > >>>>>> particular everything related to client part (job > submission, > > > > > >> workflow > > > > > >>>>>> management, callbacks on submission/success/failure, etc). > > > > > >>>>>> Something I can't find anywhere is also how to query Flink > > > > > >>>> states..would it > > > > > >>>>>> be possible to have something like the Presto UI [1]? Does > > Flink > > > > > >>>> implement > > > > > >>>>>> some sort of query queuing? I heard about a query proxy > server > > > > but I > > > > > >>>> don't > > > > > >>>>>> know if there's a will to push in that direction. > > > > > >>>>>> For Stateful Functions it would be nice to deeply compare > the > > > taxi > > > > > >>>> driver > > > > > >>>>>> solution with a more common implementation (i.e. using a > > > database > > > > to > > > > > >>>>>> persist the legal data..is it safe to keep them as a Flink > > > > state?). > > > > > >>>>>> [1] > > > > > >>>> > > > > > https://www.tutorialspoint.com/apache_presto/images/web_interface.jpg > > > > > >>>>>> > > > > > >>>>>> Best, > > > > > >>>>>> Flavio > > > > > >>>>>> > > > > > >>>>>> On Tue, Mar 3, 2020 at 10:47 AM Jeff Zhang < > zjf...@gmail.com> > > > > > >> wrote: > > > > > >>>>>> > > > > > >>>>>>> +1 for this proposal. I am preparing some articles for how > > to > > > > use > > > > > >>>> Flink > > > > > >>>>>> on > > > > > >>>>>>> Zeppelin, although it is not closely related with this > topic, > > > but > > > > > >>>> should > > > > > >>>>>> be > > > > > >>>>>>> helpful for users to get started with Flink. > > > > > >>>>>>> > > > > > >>>>>>> Till Rohrmann <trohrm...@apache.org> 于2020年3月3日周二 > 下午5:39写道： > > > > > >>>>>>> > > > > > >>>>>>>> I like the idea. +1 from my side. > > > > > >>>>>>>> > > > > > >>>>>>>> Potential topics: > > > > > >>>>>>>> - Scheduling > > > > > >>>>>>>> - Cluster partitions > > > > > >>>>>>>> - Memory configuration > > > > > >>>>>>>> - Recovery > > > > > >>>>>>>> > > > > > >>>>>>>> Cheers, > > > > > >>>>>>>> Till > > > > > >>>>>>>> > > > > > >>>>>>>> On Tue, Mar 3, 2020 at 3:56 AM Xintong Song < > > > > > >> tonysong...@gmail.com> > > > > > >>>>>>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>>> Big +1. Thanks for the idea, Arvid. > > > > > >>>>>>>>> > > > > > >>>>>>>>> I'd be excited to read such blogs. > > > > > >>>>>>>>> > > > > > >>>>>>>>> And we would also be happy to contribute some contents on > > the > > > > > >>> newest > > > > > >>>>>>>>> efforts from our team. > > > > > >>>>>>>>> Potential topics: > > > > > >>>>>>>>> - Memory configuration > > > > > >>>>>>>>> - Active Kubernetes integration > > > > > >>>>>>>>> - GPU support > > > > > >>>>>>>>> - Pluggable (dynamic) slot allocation > > > > > >>>>>>>>> > > > > > >>>>>>>>> Thank you~ > > > > > >>>>>>>>> > > > > > >>>>>>>>> Xintong Song > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> On Tue, Mar 3, 2020 at 9:59 AM Benchao Li < > > > libenc...@gmail.com > > > > > > > > > > >>>>>> wrote: > > > > > >>>>>>>>> > > > > > >>>>>>>>>> +1 for this proposal. As a contributor, it would be very > > > > helpful > > > > > >>> to > > > > > >>>>>>>> have > > > > > >>>>>>>>>> such blogs for us to understand status and future of > > Flink. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Robert Metzger <rmetz...@apache.org> 于2020年3月3日周二 > > 上午6:00写道： > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> I would be excited to read such a blog (can I request > > > topics? > > > > > >> :) > > > > > >>>>>> ) > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> We could start very low key by using our wiki's blog > > > feature: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewrecentblogposts.action?key=FLINK > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> On Mon, Mar 2, 2020 at 8:26 PM Stephan Ewen < > > > > se...@apache.org> > > > > > >>>>>>>> wrote: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> Great idea, but I also second Seth's comment to > separate > > > > this > > > > > >>>>>> in > > > > > >>>>>>> a > > > > > >>>>>>>>>> clear > > > > > >>>>>>>>>>>> way. It's easy to confuse new / potential users. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> On Mon, Mar 2, 2020 at 8:15 PM Seth Wiesman < > > > > > >>>>>> sjwies...@gmail.com > > > > > >>>>>>>> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> +1 on the idea. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> My only request would be they are clearly marked as > > being > > > > > >>>>>> about > > > > > >>>>>>>>>>>> internals / > > > > > >>>>>>>>>>>>> for advanced users to not give typical users the > wrong > > > > > >>>>>>> impression > > > > > >>>>>>>>>> about > > > > > >>>>>>>>>>>> how > > > > > >>>>>>>>>>>>> much they need to understand to use Flink. Nico's > > network > > > > > >>>>>> stack > > > > > >>>>>>>>> blog > > > > > >>>>>>>>>>> post > > > > > >>>>>>>>>>>>> does this well[1]. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Seth > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> [1] > > > > > >>>>>>> > https://flink.apache.org/2019/06/05/flink-network-stack.html > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Mon, Mar 2, 2020 at 10:39 AM Ufuk Celebi < > > > > u...@apache.org> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> I'd be happy to read such a blog. Big +1 as a > > potential > > > > > >>>>>>> reader. > > > > > >>>>>>>>> ;-) > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> – Ufuk > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> On Mon, Mar 2, 2020 at 11:53 AM Arvid Heise < > > > > > >>>>>>>> ar...@ververica.com > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Dear devs, > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> development speed of Flink has steadily increased. > > Lots > > > > > >>>>>> of > > > > > >>>>>>>> new > > > > > >>>>>>>>>>>> concepts > > > > > >>>>>>>>>>>>>> are > > > > > >>>>>>>>>>>>>>> introduced and technical debt removed. However, > it's > > > hard > > > > > >>>>>>> to > > > > > >>>>>>>>> keep > > > > > >>>>>>>>>>>> track > > > > > >>>>>>>>>>>>>> of > > > > > >>>>>>>>>>>>>>> these things if you are not directly involved. > > > Especially > > > > > >>>>>>> for > > > > > >>>>>>>>> new > > > > > >>>>>>>>>>>>>>> contributors, it's often not easy to know what the > > best > > > > > >>>>>>>>> practices > > > > > >>>>>>>>>>> are > > > > > >>>>>>>>>>>>> or > > > > > >>>>>>>>>>>>>> if > > > > > >>>>>>>>>>>>>>> there are related work streams going on. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> In the runtime team, we had the idea to set up a > dev > > > blog > > > > > >>>>>>>> where > > > > > >>>>>>>>>> we > > > > > >>>>>>>>>>>>> could > > > > > >>>>>>>>>>>>>>> introduce newest developments. The scope should be > > > expert > > > > > >>>>>>>> users > > > > > >>>>>>>>>>> that > > > > > >>>>>>>>>>>>>>> contribute to the project. Of course, some articles > > may > > > > > >>>>>>> have > > > > > >>>>>>>> a > > > > > >>>>>>>>>>>> broader > > > > > >>>>>>>>>>>>>>> scope and even be linked from release notes. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Examples from our team to give a more specific > idea: > > > > > >>>>>>>>>>>>>>> * Deprecated checkpoint lock and mailbox model > > > > > >>>>>>>>>>>>>>> * Revised interface for two phase commit sinks and > > new > > > > > >>>>>> JDBC > > > > > >>>>>>>>> sink > > > > > >>>>>>>>>>>>>>> * N-ary input operators > > > > > >>>>>>>>>>>>>>> * Unaligned checkpoints > > > > > >>>>>>>>>>>>>>> * Operator factories > > > > > >>>>>>>>>>>>>>> * Plugins > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> These articles would be less formal than a FLIP > (but > > > > > >>>>>> could > > > > > >>>>>>>> link > > > > > >>>>>>>>>> to > > > > > >>>>>>>>>>>> them > > > > > >>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>> avoid redundancy) and focus more on how other > > > developers > > > > > >>>>>>> are > > > > > >>>>>>>>>>> actually > > > > > >>>>>>>>>>>>>>> impacted by the changes. It can also be used to > share > > > > > >>>>>>>>> experiences > > > > > >>>>>>>>>>>>> during > > > > > >>>>>>>>>>>>>>> the implementation. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> We would like to ask the other teams if they see a > > > > > >>>>>> benefit > > > > > >>>>>>> of > > > > > >>>>>>>>>> such > > > > > >>>>>>>>>>> a > > > > > >>>>>>>>>>>>> blog > > > > > >>>>>>>>>>>>>>> and would like to contribute. Bonus points if you > > could > > > > > >>>>>>>> provide > > > > > >>>>>>>>>>>>> examples > > > > > >>>>>>>>>>>>>> of > > > > > >>>>>>>>>>>>>>> your topics. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Arvid > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> -- > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Benchao Li > > > > > >>>>>>>>>> School of Electronics Engineering and Computer Science, > > > Peking > > > > > >>>>>>>> University > > > > > >>>>>>>>>> Tel:+86-15650713730 > > > > > >>>>>>>>>> Email: libenc...@gmail.com; libenc...@pku.edu.cn > > > > > >>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> -- > > > > > >>>>>>> Best Regards > > > > > >>>>>>> > > > > > >>>>>>> Jeff Zhang > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> -- > > > > > >>>>> Best, Jingsong Lee > > > > > >>>> > > > > > >>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > > > > > >
