Thanks a lot for sharing the code with the community Yadong! It looks really cool and I also want to give it a try to see how easy it is to start Flink with it.
If it is already implemented and working, we could also think about adding it to Flink and add a feature flag to switch between the old and new web UI. We could think about enabling it by default to give it more user exposure. After being confident and users having no complaints, we could think about deprecating the old web UI and eventually to drop it. Of course, initially we should give it a thorough review. Cheers, Till On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fab...@zalando.de> wrote: > Hi Yadong, this is awesome, thx for the code! I will try it out on our > infrastructure and will post my feedback here, latest next week. > > I will also check if my ideas for FLINK-10707 > <https://issues.apache.org/jira/browse/FLINK-10707> are doable with your > code since this was what pushed this discussion initially. > > Cheers > > -- > > > *Fabian WollertZalando SE* > > E-Mail: fab...@zalando.de > > > Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <vthink...@gmail.com > >: > > > Hi Fabian, Till, and Robert > > > > Thank you for your attention to this matter, I just push our codes to > > github: https://github.com/vthinkxie/flink-runtime-web. > > > > You can start the project by following the guidelines > > <https://github.com/vthinkxie/flink-runtime-web#development--debugging> > > (just > > run `npm install && npm run proxy`), just feel free to give any comments > :) > > > > If I missed anything please let me know. Look forward to your feedback > and > > suggestions soon. > > > > Best regards > > Yadong > > > > > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fab...@zalando.de> wrote: > > > > > Hi Yadong, this looks awesome. is there any chance you can already > share > > > the code of the new web UI, so we can take a look at what you guys > build > > > there? I think that would speed up the discussion. If there is already > a > > > fully fledged new Version with everything updated out there, and its > even > > > battle tested in production already, that sounds like the way to go for > > me. > > > > > > i started aside from this discussion here (to strengthen and learn some > > > new React stuff) my attempt on the React version already, for whoever > is > > > curious, you can check it out here: > > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base url > in > > > actions.js > > > < > > > https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8 > > >, > > > npm install and then npm start) ... i just started to convert the first > > > simple pages, so dont expect the whole UI yet, its just a start ... > > > > > > Cheers > > > > > > -- > > > > > > > > > *Fabian WollertZalando SE* > > > > > > E-Mail: fab...@zalando.de > > > > > > > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <zjf...@gmail.com > >: > > > > > >> +1 for the proposal of Yadong. I'd like to add one more comment: > > >> > > >> We should provide more development guidance when developing the next > > >> dashboard via angular 7. As Till mentioned before, one big concern is > > lack > > >> of frontend guys in flink community, I believe more development > guidance > > >> will attract more frontend guys to contribute to Flink web ui, and > also > > be > > >> helpful for backend guys who want to add something to dashboard. > > >> > > >> > > >> > > >> Shaoxuan Wang <wshaox...@gmail.com>于2018年11月2日周五 上午10:06写道： > > >> > > >> > +1 to keep the web UI stack to Angular. This will make the stack > > >> upgrading > > >> > much easier and won't break the on-going efforts already made > towards > > >> the > > >> > improvements of Flink Web UI. > > >> > > > >> > Till, Robert, and Fabian, > > >> > Please take a look at the upgrading plan from Angular 1.x to 7.0 > > >> proposed > > >> > by Yadong. Any feedback is appreciated. > > >> > > > >> > Regards, > > >> > Shaoxuan > > >> > > > >> > > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vthink...@gmail.com> > > wrote: > > >> > > > >> > > For your interest, here are some snapshots of Flink web UI (with > > >> angular > > >> > > 7.0) we recently reconstructed. > > >> > > [image: snapshot.001.jpeg] > > >> > > > > >> > > > > >> > > > >> > > > https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing > > >> > > > > >> > > We choose Angular 7.0 due to the following reasons: > > >> > > > > >> > > 1. It is easy to migrate the current codebase to Angular 7.0 > > >> following > > >> > > the official guide. > > >> > > 2. It is easy for current Web UI developers who are experienced > > >> with > > >> > > Angular 1.x to get familiar with 7.0 reading the official doc. > > >> > > 3. Migrating other than rewriting is helpful to ensure its > > >> functional > > >> > > integrity. > > >> > > 4. There are over 1 million Angular developers now (2.0 and > > above, > > >> > > still increasing rapidly), which means there are potentially > more > > >> > > developers willing to join the community. > > >> > > 5. No bower or complex node / webpack configuration needed > (only > > >> > > angular cli). > > >> > > > > >> > > We split Angular 1.x and 7.0 versions into two folders based on > the > > >> > > following consideration: > > >> > > > > >> > > 1. 1.x version could be the default page because there is no > > >> > > breakchange for both users and developers. > > >> > > 2. Make the upgrade progressive. Users can switch between two > > >> versions > > >> > > by clicking the button on the top right corner. > > >> > > 3. Reduce maintenance costs and keep the performance of web > > without > > >> > > bootstrap both 1.x and 7.0 versions in the same page. > > >> > > 4. Won't interrupt the current development process. The code > can > > >> still > > >> > > be submitted to 1.x version folder, we could help to sync all > > >> > features and > > >> > > bug fix to 7.0 version if needed. > > >> > > 5. When the 7.0 version is ready and stable, it is easy to > change > > >> > > default page to it, and remove the old version. > > >> > > > > >> > > [image: test.jpg] > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing > > >> > > > > >> > > Here is the brief list of what we have done: > > >> > > > > >> > > 1. Create a new folder named next-web-dashboard in > > >> flink-runtime-web. > > >> > > 2. Migrate all Angular 1.x controllers, services, config etc > one > > by > > >> > > one into 7.0 version with ngMigration. > > >> > > 3. Modify frontend build scripts in order to package two > versions > > >> > > together. > > >> > > 4. All works listed above are limited in the flink-runtime-web > > >> folder. > > >> > > One folder was added and no file or folder was deleted. > > >> > > > > >> > > The new version of the Web UI is running stable for several months > > in > > >> > > Alibaba. We are happy to contribute it to the Flink community if > > >> everyone > > >> > > like the work. > > >> > > > > >> > > Best > > >> > > Yadong > > >> > > > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann < > trohrm...@apache.org > > > > > >> > > wrote: > > >> > > > > >> > >> I agree with Chesnay that the web UI is already fairly isolated > in > > >> > Flink's > > >> > >> code base (see flink-runtime-web module). Moreover, imo it is > very > > >> > >> important that Flink comes with a nice web UI which works out of > > the > > >> box > > >> > >> because it lowers the entrance hurdle massively. What one could > > >> argue is > > >> > >> to > > >> > >> make it easier to plug in your own web UI if you want to develop > > it. > > >> > But I > > >> > >> would see this as an orthogonal issue. > > >> > >> > > >> > >> I would actually argue against Robert's proposal to make a grand > > >> rewrite > > >> > >> if > > >> > >> this is by any means possible. Experience shows that these kind > of > > >> > efforts > > >> > >> are usually set up to fail or take super long. If possible, then > I > > >> would > > >> > >> rather prefer to convert one component at a time to React. But > I'm > > >> not > > >> > an > > >> > >> expert here and cannot tell whether this is possible or not. > > >> > >> > > >> > >> What we would need is active help from the community to review > > these > > >> > >> changes. I think that is the crucial piece to make such an effort > > >> work > > >> > or > > >> > >> fail. > > >> > >> > > >> > >> Cheers, > > >> > >> Till > > >> > >> > > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler < > > >> ches...@apache.org> > > >> > >> wrote: > > >> > >> > > >> > >> > The WebUI goes entirely through the REST API. > > >> > >> > > > >> > >> > I don't see how moving it out would make it easier to > > >> > >> develop/contribute. > > >> > >> > > > >> > >> > The UI is already encapsulated in it's own module > > >> (flink-runtime-web) > > >> > >> > and changes can be made to it independently. > > >> > >> > Historically the issue was simply that contributions to the UI > > were > > >> > not > > >> > >> > reviewed in a timely manner / not at all, which mind you > > >> > >> > applies to virtually all components. This also won't change by > > >> moving > > >> > it > > >> > >> > out of Flink. > > >> > >> > > > >> > >> > Having it in Flink also guarantees that there is an > > >> > easily-discoverable > > >> > >> > version of the UI that is compatible with your Flink > > distribution. > > >> > >> > > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote: > > >> > >> > > Re: who's using the web ui > > >> > >> > > > > >> > >> > > Though many mature solutions, with a fair amount of > > >> time/resources > > >> > >> > available are likely running their own front ends, for teams > like > > >> mine > > >> > >> > which are smaller and aren't focused solely on working with > > Flink, > > >> > >> having > > >> > >> > the web ui available removes a large barrier to getting up and > > >> running > > >> > >> > quickly. > > >> > >> > > > > >> > >> > > But, with the web UI being in core Flink, it does make it a > bit > > >> more > > >> > >> of > > >> > >> > a chore to contribute. > > >> > >> > > > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy with > > >> Flink? > > >> > Is > > >> > >> > it making any calls that aren't through the REST API (and if > so, > > >> can > > >> > >> those > > >> > >> > endpoints be added to the REST API)? > > >> > >> > > > > >> > >> > > In general I'd support moving it to its own package, making > it > > >> > easier > > >> > >> to > > >> > >> > develop with your standard UI dev tools. This also allows the > web > > >> UI > > >> > to > > >> > >> be > > >> > >> > developed and released independently to core Flink. > > >> > >> > > > > >> > >> > > Dyana > > >> > >> > > > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fab...@zalando.de> > > >> wrote: > > >> > >> > >> Hi Till, I basically agree with all your points. i would > > stress > > >> the > > >> > >> > >> "dustiness" of the current architecture: the package manager > > >> used > > >> > >> > (bower) > > >> > >> > >> is deprecated since a long time, the chance for the builds > of > > >> the > > >> > >> flink > > >> > >> > web > > >> > >> > >> dashboard not working anymore is increasing every day. > > >> > >> > >> > > >> > >> > >> About the knowledge in the community: Two days is not a lot > of > > >> > time, > > >> > >> but > > >> > >> > >> interest in this topic seems to be minor anyways. Is someone > > >> using > > >> > >> the > > >> > >> > >> Flink Web Dashboard at all, or is everyone running their own > > >> custom > > >> > >> > >> solutions? Because if there is no interest in using the Web > UI > > >> AND > > >> > no > > >> > >> > one > > >> > >> > >> interested in developing, there would be no need to package > > >> this as > > >> > >> > part of > > >> > >> > >> the official Flink package, but rather develop an > independent > > >> > >> solution > > >> > >> > (I'm > > >> > >> > >> not voting for this right now, just putting it out), if at > > all. > > >> The > > >> > >> > >> official package could then just ship with the API, which > > other > > >> > >> > solutions > > >> > >> > >> can build upon. This solution could be from an agile point > of > > >> view > > >> > >> also > > >> > >> > the > > >> > >> > >> best (enhanced testing, independent and more effective dev > > >> > workflow, > > >> > >> > etc.), > > >> > >> > >> but is bad for the usage of the Flink UI, because people > need > > to > > >> > >> install > > >> > >> > >> two things individually (Flink and the web dashboard). > > >> > >> > >> > > >> > >> > >> How did the choice for Angular1 happen back then? Who was > > >> writing > > >> > the > > >> > >> > >> Dashboard in the first place? > > >> > >> > >> > > >> > >> > >> Cheers > > >> > >> > >> > > >> > >> > >> -- > > >> > >> > >> > > >> > >> > >> > > >> > >> > >> *Fabian WollertZalando SE* > > >> > >> > >> > > >> > >> > >> E-Mail: fab...@zalando.de > > >> > >> > >> > > >> > >> > >> > > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann < > > >> > >> > >> trohrm...@apache.org>: > > >> > >> > >> > > >> > >> > >>> Thanks for starting this discussion Fabian! I think our web > > UI > > >> > >> > technology > > >> > >> > >>> stack is quite dusty by now and it would be beneficial to > > think > > >> > >> about > > >> > >> > its > > >> > >> > >>> technological future. > > >> > >> > >>> > > >> > >> > >>> On the one hand, our current web UI works more or less > > reliable > > >> > and > > >> > >> > >>> changing the underlying technology has the risk of breaking > > >> > things. > > >> > >> > >>> Moreover, there might be the risk that the newly chosen > > >> technology > > >> > >> > will be > > >> > >> > >>> deprecated at some point in time as well. > > >> > >> > >>> > > >> > >> > >>> On the other hand, we don't have much Angular 1 knowledge > in > > >> the > > >> > >> > community > > >> > >> > >>> and extending the web UI is, thus, quite hard at the > moment. > > >> Maybe > > >> > >> by > > >> > >> > using > > >> > >> > >>> some newer web technologies we might be able to attract > more > > >> > people > > >> > >> > with a > > >> > >> > >>> web technology background to join the community. > > >> > >> > >>> > > >> > >> > >>> The lack of people working on the web UI is for me the > > biggest > > >> > >> problem > > >> > >> > I > > >> > >> > >>> would like to address. If there is interest in the web UI, > > then > > >> > I'm > > >> > >> > quite > > >> > >> > >>> sure that we will be able to even migrate to some other > > >> technology > > >> > >> in > > >> > >> > the > > >> > >> > >>> future. The next important issue for me is to do the change > > >> > >> > incrementally > > >> > >> > >>> if possible. Ideally we never break the web UI in the > process > > >> of > > >> > >> > migrating > > >> > >> > >>> to a new technology. I'm not an expert here so it might or > > >> might > > >> > >> not be > > >> > >> > >>> possible. But if it is, then we should design the > > >> implementation > > >> > >> steps > > >> > >> > in > > >> > >> > >>> such a way. > > >> > >> > >>> > > >> > >> > >>> Cheers, > > >> > >> > >>> Till > > >> > >> > >>> > > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert < > > >> fab...@zalando.de > > >> > > > > >> > >> > wrote: > > >> > >> > >>> > > >> > >> > >>>> Hi everyone, > > >> > >> > >>>> > > >> > >> > >>>> in this email thread > > >> > >> > >>>> < > > >> > >> > >>>> > > >> > >> > >>> > > >> > >> > > > >> > >> > > >> > > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html > > >> > >> > >>>> and the tickets FLINK-10705 > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and > > >> > >> FLINK-10706 > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the > > >> > discussion > > >> > >> > came > > >> > >> > >>> up > > >> > >> > >>>> whether to change the underlying architecture of Flink's > Web > > >> > >> Dashboard > > >> > >> > >>> from > > >> > >> > >>>> Angular1 to something else. This email thread should be > > >> solely to > > >> > >> > discuss > > >> > >> > >>>> the pro's and con's of this, and what could be the target > > >> > >> > architecture. > > >> > >> > >>>> > > >> > >> > >>>> My choice would be React. Personally I agree with Till's > > >> comments > > >> > >> in > > >> > >> > the > > >> > >> > >>>> ticket, Angular 1 being basically outdated and is not > > having a > > >> > >> large > > >> > >> > >>>> following anymore. From my experience the choice between > > >> Angular > > >> > >> 2-7 > > >> > >> > or > > >> > >> > >>>> React is subjective, you can get things done with both. I > > >> > >> personally > > >> > >> > only > > >> > >> > >>>> have experience with React, so I personally would be > faster > > >> to > > >> > >> > develop > > >> > >> > >>>> with this one. I currently have not planned to learn > Angular > > >> as > > >> > >> well > > >> > >> > >>> (being > > >> > >> > >>>> a more backend focused developer in general) so if the > > >> decision > > >> > >> would > > >> > >> > be > > >> > >> > >>> to > > >> > >> > >>>> go with Angular, i would be unfortunately out of this > rework > > >> of > > >> > the > > >> > >> > Flink > > >> > >> > >>>> Dashboard most certainly. > > >> > >> > >>>> > > >> > >> > >>>> Additionally i would like to get rid of bower, since its > > >> > officially > > >> > >> > >>>> deprecated < > > >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>. > > >> > >> > >>>> my > > >> > >> > >>>> idea would be to just use a create-react-app package with > > npm > > >> and > > >> > >> > webpack > > >> > >> > >>>> under the hood. no need for additional lib's here imho. > But > > >> > again: > > >> > >> > thats > > >> > >> > >>>> mostly what i've been working with recently, so thats a > > >> > subjective > > >> > >> > >>> point. I > > >> > >> > >>>> could imagine getting used to yarn in the future as well. > > >> > >> > >>>> > > >> > >> > >>>> Cheers > > >> > >> > >>>> Fabian > > >> > >> > >>>> > > >> > >> > >>>> -- > > >> > >> > >>>> > > >> > >> > >>>> > > >> > >> > >>>> *Fabian WollertZalando SE* > > >> > >> > >>>> > > >> > >> > >>>> E-Mail: fab...@zalando.de > > >> > >> > >>>> > > >> > >> > > > >> > >> > > > >> > >> > > >> > > > > >> > > > > >> > > -- > > >> > > Yadong Xie > > >> > > Ministry of Education Key Lab for Intelligent Networks and Network > > >> > Security > > >> > > Dept. of Computer Science and Technology > > >> > > School of electronic information and engineering > > >> > > Xi'an Jiaotong University > > >> > > Xi'an, 710049, P.R. China > > >> > > Email: vthink...@gmail.com > > >> > > > > >> > > > >> > > > > > >
