Till, Yes, it is a good idea to have a feature flag to switch the web UI before we completely deprecate the old one.
Yadong, It seems that everyone likes the new web UI. Can you please open a master Jira and start to merge the code to Flink master. What do you think? Regards, Shaoxuan On Mon, Nov 5, 2018 at 7:04 PM Till Rohrmann <trohrm...@apache.org> wrote: > Thanks a lot for sharing the code with the community Yadong! > > It looks really cool and I also want to give it a try to see how easy it is > to start Flink with it. > > If it is already implemented and working, we could also think about adding > it to Flink and add a feature flag to switch between the old and new web > UI. We could think about enabling it by default to give it more user > exposure. After being confident and users having no complaints, we could > think about deprecating the old web UI and eventually to drop it. Of > course, initially we should give it a thorough review. > > Cheers, > Till > > On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fab...@zalando.de> wrote: > > > Hi Yadong, this is awesome, thx for the code! I will try it out on our > > infrastructure and will post my feedback here, latest next week. > > > > I will also check if my ideas for FLINK-10707 > > <https://issues.apache.org/jira/browse/FLINK-10707> are doable with your > > code since this was what pushed this discussion initially. > > > > Cheers > > > > -- > > > > > > *Fabian WollertZalando SE* > > > > E-Mail: fab...@zalando.de > > > > > > Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie < > vthink...@gmail.com > > >: > > > > > Hi Fabian, Till, and Robert > > > > > > Thank you for your attention to this matter, I just push our codes to > > > github: https://github.com/vthinkxie/flink-runtime-web. > > > > > > You can start the project by following the guidelines > > > <https://github.com/vthinkxie/flink-runtime-web#development--debugging > > > > > (just > > > run `npm install && npm run proxy`), just feel free to give any > comments > > :) > > > > > > If I missed anything please let me know. Look forward to your feedback > > and > > > suggestions soon. > > > > > > Best regards > > > Yadong > > > > > > > > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fab...@zalando.de> > wrote: > > > > > > > Hi Yadong, this looks awesome. is there any chance you can already > > share > > > > the code of the new web UI, so we can take a look at what you guys > > build > > > > there? I think that would speed up the discussion. If there is > already > > a > > > > fully fledged new Version with everything updated out there, and its > > even > > > > battle tested in production already, that sounds like the way to go > for > > > me. > > > > > > > > i started aside from this discussion here (to strengthen and learn > some > > > > new React stuff) my attempt on the React version already, for whoever > > is > > > > curious, you can check it out here: > > > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base > url > > in > > > > actions.js > > > > < > > > > > > https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8 > > > >, > > > > npm install and then npm start) ... i just started to convert the > first > > > > simple pages, so dont expect the whole UI yet, its just a start ... > > > > > > > > Cheers > > > > > > > > -- > > > > > > > > > > > > *Fabian WollertZalando SE* > > > > > > > > E-Mail: fab...@zalando.de > > > > > > > > > > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang < > zjf...@gmail.com > > >: > > > > > > > >> +1 for the proposal of Yadong. I'd like to add one more comment: > > > >> > > > >> We should provide more development guidance when developing the next > > > >> dashboard via angular 7. As Till mentioned before, one big concern > is > > > lack > > > >> of frontend guys in flink community, I believe more development > > guidance > > > >> will attract more frontend guys to contribute to Flink web ui, and > > also > > > be > > > >> helpful for backend guys who want to add something to dashboard. > > > >> > > > >> > > > >> > > > >> Shaoxuan Wang <wshaox...@gmail.com>于2018年11月2日周五 上午10:06写道： > > > >> > > > >> > +1 to keep the web UI stack to Angular. This will make the stack > > > >> upgrading > > > >> > much easier and won't break the on-going efforts already made > > towards > > > >> the > > > >> > improvements of Flink Web UI. > > > >> > > > > >> > Till, Robert, and Fabian, > > > >> > Please take a look at the upgrading plan from Angular 1.x to 7.0 > > > >> proposed > > > >> > by Yadong. Any feedback is appreciated. > > > >> > > > > >> > Regards, > > > >> > Shaoxuan > > > >> > > > > >> > > > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vthink...@gmail.com> > > > wrote: > > > >> > > > > >> > > For your interest, here are some snapshots of Flink web UI (with > > > >> angular > > > >> > > 7.0) we recently reconstructed. > > > >> > > [image: snapshot.001.jpeg] > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing > > > >> > > > > > >> > > We choose Angular 7.0 due to the following reasons: > > > >> > > > > > >> > > 1. It is easy to migrate the current codebase to Angular 7.0 > > > >> following > > > >> > > the official guide. > > > >> > > 2. It is easy for current Web UI developers who are > experienced > > > >> with > > > >> > > Angular 1.x to get familiar with 7.0 reading the official > doc. > > > >> > > 3. Migrating other than rewriting is helpful to ensure its > > > >> functional > > > >> > > integrity. > > > >> > > 4. There are over 1 million Angular developers now (2.0 and > > > above, > > > >> > > still increasing rapidly), which means there are potentially > > more > > > >> > > developers willing to join the community. > > > >> > > 5. No bower or complex node / webpack configuration needed > > (only > > > >> > > angular cli). > > > >> > > > > > >> > > We split Angular 1.x and 7.0 versions into two folders based on > > the > > > >> > > following consideration: > > > >> > > > > > >> > > 1. 1.x version could be the default page because there is no > > > >> > > breakchange for both users and developers. > > > >> > > 2. Make the upgrade progressive. Users can switch between two > > > >> versions > > > >> > > by clicking the button on the top right corner. > > > >> > > 3. Reduce maintenance costs and keep the performance of web > > > without > > > >> > > bootstrap both 1.x and 7.0 versions in the same page. > > > >> > > 4. Won't interrupt the current development process. The code > > can > > > >> still > > > >> > > be submitted to 1.x version folder, we could help to sync all > > > >> > features and > > > >> > > bug fix to 7.0 version if needed. > > > >> > > 5. When the 7.0 version is ready and stable, it is easy to > > change > > > >> > > default page to it, and remove the old version. > > > >> > > > > > >> > > [image: test.jpg] > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing > > > >> > > > > > >> > > Here is the brief list of what we have done: > > > >> > > > > > >> > > 1. Create a new folder named next-web-dashboard in > > > >> flink-runtime-web. > > > >> > > 2. Migrate all Angular 1.x controllers, services, config etc > > one > > > by > > > >> > > one into 7.0 version with ngMigration. > > > >> > > 3. Modify frontend build scripts in order to package two > > versions > > > >> > > together. > > > >> > > 4. All works listed above are limited in the > flink-runtime-web > > > >> folder. > > > >> > > One folder was added and no file or folder was deleted. > > > >> > > > > > >> > > The new version of the Web UI is running stable for several > months > > > in > > > >> > > Alibaba. We are happy to contribute it to the Flink community if > > > >> everyone > > > >> > > like the work. > > > >> > > > > > >> > > Best > > > >> > > Yadong > > > >> > > > > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann < > > trohrm...@apache.org > > > > > > > >> > > wrote: > > > >> > > > > > >> > >> I agree with Chesnay that the web UI is already fairly isolated > > in > > > >> > Flink's > > > >> > >> code base (see flink-runtime-web module). Moreover, imo it is > > very > > > >> > >> important that Flink comes with a nice web UI which works out > of > > > the > > > >> box > > > >> > >> because it lowers the entrance hurdle massively. What one could > > > >> argue is > > > >> > >> to > > > >> > >> make it easier to plug in your own web UI if you want to > develop > > > it. > > > >> > But I > > > >> > >> would see this as an orthogonal issue. > > > >> > >> > > > >> > >> I would actually argue against Robert's proposal to make a > grand > > > >> rewrite > > > >> > >> if > > > >> > >> this is by any means possible. Experience shows that these kind > > of > > > >> > efforts > > > >> > >> are usually set up to fail or take super long. If possible, > then > > I > > > >> would > > > >> > >> rather prefer to convert one component at a time to React. But > > I'm > > > >> not > > > >> > an > > > >> > >> expert here and cannot tell whether this is possible or not. > > > >> > >> > > > >> > >> What we would need is active help from the community to review > > > these > > > >> > >> changes. I think that is the crucial piece to make such an > effort > > > >> work > > > >> > or > > > >> > >> fail. > > > >> > >> > > > >> > >> Cheers, > > > >> > >> Till > > > >> > >> > > > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler < > > > >> ches...@apache.org> > > > >> > >> wrote: > > > >> > >> > > > >> > >> > The WebUI goes entirely through the REST API. > > > >> > >> > > > > >> > >> > I don't see how moving it out would make it easier to > > > >> > >> develop/contribute. > > > >> > >> > > > > >> > >> > The UI is already encapsulated in it's own module > > > >> (flink-runtime-web) > > > >> > >> > and changes can be made to it independently. > > > >> > >> > Historically the issue was simply that contributions to the > UI > > > were > > > >> > not > > > >> > >> > reviewed in a timely manner / not at all, which mind you > > > >> > >> > applies to virtually all components. This also won't change > by > > > >> moving > > > >> > it > > > >> > >> > out of Flink. > > > >> > >> > > > > >> > >> > Having it in Flink also guarantees that there is an > > > >> > easily-discoverable > > > >> > >> > version of the UI that is compatible with your Flink > > > distribution. > > > >> > >> > > > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote: > > > >> > >> > > Re: who's using the web ui > > > >> > >> > > > > > >> > >> > > Though many mature solutions, with a fair amount of > > > >> time/resources > > > >> > >> > available are likely running their own front ends, for teams > > like > > > >> mine > > > >> > >> > which are smaller and aren't focused solely on working with > > > Flink, > > > >> > >> having > > > >> > >> > the web ui available removes a large barrier to getting up > and > > > >> running > > > >> > >> > quickly. > > > >> > >> > > > > > >> > >> > > But, with the web UI being in core Flink, it does make it a > > bit > > > >> more > > > >> > >> of > > > >> > >> > a chore to contribute. > > > >> > >> > > > > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy > with > > > >> Flink? > > > >> > Is > > > >> > >> > it making any calls that aren't through the REST API (and if > > so, > > > >> can > > > >> > >> those > > > >> > >> > endpoints be added to the REST API)? > > > >> > >> > > > > > >> > >> > > In general I'd support moving it to its own package, making > > it > > > >> > easier > > > >> > >> to > > > >> > >> > develop with your standard UI dev tools. This also allows the > > web > > > >> UI > > > >> > to > > > >> > >> be > > > >> > >> > developed and released independently to core Flink. > > > >> > >> > > > > > >> > >> > > Dyana > > > >> > >> > > > > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fab...@zalando.de> > > > >> wrote: > > > >> > >> > >> Hi Till, I basically agree with all your points. i would > > > stress > > > >> the > > > >> > >> > >> "dustiness" of the current architecture: the package > manager > > > >> used > > > >> > >> > (bower) > > > >> > >> > >> is deprecated since a long time, the chance for the builds > > of > > > >> the > > > >> > >> flink > > > >> > >> > web > > > >> > >> > >> dashboard not working anymore is increasing every day. > > > >> > >> > >> > > > >> > >> > >> About the knowledge in the community: Two days is not a > lot > > of > > > >> > time, > > > >> > >> but > > > >> > >> > >> interest in this topic seems to be minor anyways. Is > someone > > > >> using > > > >> > >> the > > > >> > >> > >> Flink Web Dashboard at all, or is everyone running their > own > > > >> custom > > > >> > >> > >> solutions? Because if there is no interest in using the > Web > > UI > > > >> AND > > > >> > no > > > >> > >> > one > > > >> > >> > >> interested in developing, there would be no need to > package > > > >> this as > > > >> > >> > part of > > > >> > >> > >> the official Flink package, but rather develop an > > independent > > > >> > >> solution > > > >> > >> > (I'm > > > >> > >> > >> not voting for this right now, just putting it out), if at > > > all. > > > >> The > > > >> > >> > >> official package could then just ship with the API, which > > > other > > > >> > >> > solutions > > > >> > >> > >> can build upon. This solution could be from an agile point > > of > > > >> view > > > >> > >> also > > > >> > >> > the > > > >> > >> > >> best (enhanced testing, independent and more effective dev > > > >> > workflow, > > > >> > >> > etc.), > > > >> > >> > >> but is bad for the usage of the Flink UI, because people > > need > > > to > > > >> > >> install > > > >> > >> > >> two things individually (Flink and the web dashboard). > > > >> > >> > >> > > > >> > >> > >> How did the choice for Angular1 happen back then? Who was > > > >> writing > > > >> > the > > > >> > >> > >> Dashboard in the first place? > > > >> > >> > >> > > > >> > >> > >> Cheers > > > >> > >> > >> > > > >> > >> > >> -- > > > >> > >> > >> > > > >> > >> > >> > > > >> > >> > >> *Fabian WollertZalando SE* > > > >> > >> > >> > > > >> > >> > >> E-Mail: fab...@zalando.de > > > >> > >> > >> > > > >> > >> > >> > > > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann < > > > >> > >> > >> trohrm...@apache.org>: > > > >> > >> > >> > > > >> > >> > >>> Thanks for starting this discussion Fabian! I think our > web > > > UI > > > >> > >> > technology > > > >> > >> > >>> stack is quite dusty by now and it would be beneficial to > > > think > > > >> > >> about > > > >> > >> > its > > > >> > >> > >>> technological future. > > > >> > >> > >>> > > > >> > >> > >>> On the one hand, our current web UI works more or less > > > reliable > > > >> > and > > > >> > >> > >>> changing the underlying technology has the risk of > breaking > > > >> > things. > > > >> > >> > >>> Moreover, there might be the risk that the newly chosen > > > >> technology > > > >> > >> > will be > > > >> > >> > >>> deprecated at some point in time as well. > > > >> > >> > >>> > > > >> > >> > >>> On the other hand, we don't have much Angular 1 knowledge > > in > > > >> the > > > >> > >> > community > > > >> > >> > >>> and extending the web UI is, thus, quite hard at the > > moment. > > > >> Maybe > > > >> > >> by > > > >> > >> > using > > > >> > >> > >>> some newer web technologies we might be able to attract > > more > > > >> > people > > > >> > >> > with a > > > >> > >> > >>> web technology background to join the community. > > > >> > >> > >>> > > > >> > >> > >>> The lack of people working on the web UI is for me the > > > biggest > > > >> > >> problem > > > >> > >> > I > > > >> > >> > >>> would like to address. If there is interest in the web > UI, > > > then > > > >> > I'm > > > >> > >> > quite > > > >> > >> > >>> sure that we will be able to even migrate to some other > > > >> technology > > > >> > >> in > > > >> > >> > the > > > >> > >> > >>> future. The next important issue for me is to do the > change > > > >> > >> > incrementally > > > >> > >> > >>> if possible. Ideally we never break the web UI in the > > process > > > >> of > > > >> > >> > migrating > > > >> > >> > >>> to a new technology. I'm not an expert here so it might > or > > > >> might > > > >> > >> not be > > > >> > >> > >>> possible. But if it is, then we should design the > > > >> implementation > > > >> > >> steps > > > >> > >> > in > > > >> > >> > >>> such a way. > > > >> > >> > >>> > > > >> > >> > >>> Cheers, > > > >> > >> > >>> Till > > > >> > >> > >>> > > > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert < > > > >> fab...@zalando.de > > > >> > > > > > >> > >> > wrote: > > > >> > >> > >>> > > > >> > >> > >>>> Hi everyone, > > > >> > >> > >>>> > > > >> > >> > >>>> in this email thread > > > >> > >> > >>>> < > > > >> > >> > >>>> > > > >> > >> > >>> > > > >> > >> > > > > >> > >> > > > >> > > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html > > > >> > >> > >>>> and the tickets FLINK-10705 > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and > > > >> > >> FLINK-10706 > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the > > > >> > discussion > > > >> > >> > came > > > >> > >> > >>> up > > > >> > >> > >>>> whether to change the underlying architecture of Flink's > > Web > > > >> > >> Dashboard > > > >> > >> > >>> from > > > >> > >> > >>>> Angular1 to something else. This email thread should be > > > >> solely to > > > >> > >> > discuss > > > >> > >> > >>>> the pro's and con's of this, and what could be the > target > > > >> > >> > architecture. > > > >> > >> > >>>> > > > >> > >> > >>>> My choice would be React. Personally I agree with Till's > > > >> comments > > > >> > >> in > > > >> > >> > the > > > >> > >> > >>>> ticket, Angular 1 being basically outdated and is not > > > having a > > > >> > >> large > > > >> > >> > >>>> following anymore. From my experience the choice between > > > >> Angular > > > >> > >> 2-7 > > > >> > >> > or > > > >> > >> > >>>> React is subjective, you can get things done with both. > I > > > >> > >> personally > > > >> > >> > only > > > >> > >> > >>>> have experience with React, so I personally would be > > faster > > > >> to > > > >> > >> > develop > > > >> > >> > >>>> with this one. I currently have not planned to learn > > Angular > > > >> as > > > >> > >> well > > > >> > >> > >>> (being > > > >> > >> > >>>> a more backend focused developer in general) so if the > > > >> decision > > > >> > >> would > > > >> > >> > be > > > >> > >> > >>> to > > > >> > >> > >>>> go with Angular, i would be unfortunately out of this > > rework > > > >> of > > > >> > the > > > >> > >> > Flink > > > >> > >> > >>>> Dashboard most certainly. > > > >> > >> > >>>> > > > >> > >> > >>>> Additionally i would like to get rid of bower, since its > > > >> > officially > > > >> > >> > >>>> deprecated < > > > >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>. > > > >> > >> > >>>> my > > > >> > >> > >>>> idea would be to just use a create-react-app package > with > > > npm > > > >> and > > > >> > >> > webpack > > > >> > >> > >>>> under the hood. no need for additional lib's here imho. > > But > > > >> > again: > > > >> > >> > thats > > > >> > >> > >>>> mostly what i've been working with recently, so thats a > > > >> > subjective > > > >> > >> > >>> point. I > > > >> > >> > >>>> could imagine getting used to yarn in the future as > well. > > > >> > >> > >>>> > > > >> > >> > >>>> Cheers > > > >> > >> > >>>> Fabian > > > >> > >> > >>>> > > > >> > >> > >>>> -- > > > >> > >> > >>>> > > > >> > >> > >>>> > > > >> > >> > >>>> *Fabian WollertZalando SE* > > > >> > >> > >>>> > > > >> > >> > >>>> E-Mail: fab...@zalando.de > > > >> > >> > >>>> > > > >> > >> > > > > >> > >> > > > > >> > >> > > > >> > > > > > >> > > > > > >> > > -- > > > >> > > Yadong Xie > > > >> > > Ministry of Education Key Lab for Intelligent Networks and > Network > > > >> > Security > > > >> > > Dept. of Computer Science and Technology > > > >> > > School of electronic information and engineering > > > >> > > Xi'an Jiaotong University > > > >> > > Xi'an, 710049, P.R. China > > > >> > > Email: vthink...@gmail.com > > > >> > > > > > >> > > > > >> > > > > > > > > > >
