Hi Team, I agree with Thomas. The main focus of this proposal was to provide a UI layer using existing REST APIs. I think we should first focus on providing a meaningful UI served by AM to user using the same existing REST APIs.
If in general, there is a agreement for the proposal, I can proceed with creating a JIRA and feature branch for the same. Thanks, Chinmay. On Wed, Jun 13, 2018 at 12:21 AM Thomas Weise <t...@apache.org> wrote: > Looks like there is some disconnect in this thread, a few replies express > concerns that the proposal IMO doesn't imply. > > There is already a REST API that backs all of the CLI functionality. Modern > UIs run in the browser, and so what was proposed (unless I misunderstood) > would be based on the same REST API and nothing extra. The only additional > thing the AM has to do is serve static files. And AFAIK it would actually > simplify the UI implementation when files and REST API have the same > origin. > > To me where files are coming from is actually a smaller piece and something > that can be augmented. But before proposing solutions outside of Apex or > additional deployment requirements, please consider the user experience. It > should be really simple to setup and use the UI. > > I would expect that a large part of the discussion evolves around the UI > functionality that helps the target persona. > > Thanks, > Thomas > > On Tue, Jun 12, 2018, 8:05 PM Shubhrajyoti Mohapatra < > sjmohapat...@gmail.com> > wrote: > > > Hi Community, > > > > I would like to propose a slightly different approach on having an UI for > > Apex. > > > > The details are below: > > > > 1. This involves having an external web-server installed along side of > > apexCli. We will basically mimic all the functionalities that apexCli > > currently provides. > > 2. We will develop the necessary REST APIs for fetch/update operations > > which apexCli currently supports. > > 3. Because most of the information and operations needed by a typical > > user are already there in apexCli, it will of great help to end users > > and > > which may increase the adoption rate. > > > > About Chinmay's proposal of having the STRAM hosting the apps, please > find > > my comments below: > > > > 1. It may increase load on STRAM if we were to provide real time > display > > of information. I am aware that apexCli fetches info from STRAM apis > but > > continuous polling can cause extra over-head. > > 2. The UI will have limited scope for the particular application. > Users > > would definitely love to have a comprehensive UI for the management of > > Apex > > Apps as a whole. > > 3. UI might be able to affect the whole application through STRAM > hence > > giving direct access to STRAM APIs might not be good idea. > > 4. Because STRAM can't have a comprehensive UI hosted inside, anyways > we > > will have to develop an external one. > > > > So I would like propose the plan on action like this. > > > > 1. We develop the wrapper UI for apexCli in the first phase and give > > user all the management options as REST APIs. > > 2. If some of information we require are not in the current apexCli > > implementation, we can add those feature to apexCli and if required to > > STRAM APIs in subsequent phases. > > 3. In future we can even remove the REST server from STRAM if we could > > provide something like MetricsStore functionality which was there in > > DataTorrent RTS. > > > > -- > > Thanks and Regards, > > Shubhrajyoti Mohapatra > > > > > > > > > > On Tue, Jun 12, 2018 at 2:49 AM Vlad Rozov <vro...@apache.org> wrote: > > > > > IMO it will be hard to provide UI that meets everyone expectation. An > > > external webUI can serve two goals > > > > > > 1. provide reference UI implementation > > > 2. ensure that REST API have all necessary entry points > > > > > > by embedding webUI directly into stram it will be tempting to shortcut > > > some calls to use stram classes directly. > > > > > > Thank you, > > > > > > Vlad > > > > > > On 6/11/18 12:18, Pramod Immaneni wrote: > > > > I too think that a comprehensive UI works better standalone but given > > > that > > > > stram already runs a webapp server, provides web services and RM > > > provides a > > > > proxy with mapped url space for it, we can extend stram to provide > > better > > > > visual output. Overall I am +1 on the idea. > > > > > > > > On Mon, Jun 11, 2018 at 8:59 AM Vlad Rozov <vro...@apache.org> > wrote: > > > > > > > >> Clarification: I am -0 on making UI part of the stram. I am +1 on > > > >> providing reference webUI application outside of the stram. > > > >> > > > >> Thank you, > > > >> > > > >> Vlad > > > >> > > > >> On 6/9/18 07:05, Vlad Rozov wrote: > > > >>> -0: stram provides REST API to query runtime information and IMO > this > > > >>> is sufficient to build an external webUI application(s). > > > >>> > > > >>> Thank you, > > > >>> > > > >>> Vlad > > > >>> > > > >>> On 6/6/18 11:17, Ambarish Pande wrote: > > > >>>> +1 For this feature. It would be really helpfull for users to > > > >>>> visualize all > > > >>>> this information and the DAG. > > > >>>> > > > >>>> +1 for ReactJS + MaterialUI > > > >>>> A great choice, given its flexibility and performance. > > > >>>> > > > >>>> I would like to contribute to this effort. > > > >>>> > > > >>>> On Wed, 6 Jun 2018 at 8:13 PM, Chinmay Kolhatkar < > > chin...@apache.org> > > > >>>> wrote: > > > >>>> > > > >>>>> Hi Community, > > > >>>>> > > > >>>>> I want to propose a feature in apex-core to have a UI component > in > > > >>>>> stram. > > > >>>>> This is influenced by how basic runtime information is shown on > UI > > by > > > >>>>> spark. > > > >>>>> > > > >>>>> This includes following features: > > > >>>>> 1. Webpage will be hosted by stram and will be available for user > > to > > > >>>>> view > > > >>>>> in one of the two ways (I'll prefer approach b): > > > >>>>> a. Hosted on a static port in which case we'll need to add a > > new > > > >>>>> servlet > > > >>>>> to stram > > > >>>>> b. The webpage will be hosted on the same port as that of > > stram > > > >>>>> webservice but at a different path. Say, http:// > > > >>>>> <stram_host>:<webservice_port>/ui > > > >>>>> > > > >>>>> 2. The webpage will be a static page and depending on the > framework > > > we > > > >>>>> chose, it can show the realtime metric data from stram. > > > >>>>> > > > >>>>> 3. There will be a categories of readonly information (maybe > > > >>>>> dynamically > > > >>>>> changing) that will be shown on the UI: > > > >>>>> a. Application level information > > > >>>>> i. Status > > > >>>>> ii. Number of tuples processed/emitted > > > >>>>> iii. Start time/ Running span > > > >>>>> iv. Stram events > > > >>>>> v. Total memory used/available > > > >>>>> vi. Number of container allocated/deployed > > > >>>>> v. Other information which is available in stram > > > >>>>> b. Logical Operator level information > > > >>>>> i. Status > > > >>>>> ii. Number of tuples processed/emitted > > > >>>>> iii. Important events related to logical operator > > > >>>>> iv. Container list in which the operator is deployed > > > >>>>> v. Any other information available in stram > > > >>>>> vi. Logical DAG View > > > >>>>> c. Container level information > > > >>>>> i. Status > > > >>>>> ii. Number of tuples processed/emitted > > > >>>>> iii. Important events related to logical operator > > > >>>>> iv. Any other information available in stram > > > >>>>> v. Physical DAG View > > > >>>>> > > > >>>>> 4. In second phase, there will be control related operations > > allowed > > > >>>>> from > > > >>>>> the UI, as follows: > > > >>>>> a. Stop/Kill application > > > >>>>> b. Stop/Kill containers > > > >>>>> c. Stack trace dump of containers > > > >>>>> d. Logs related? > > > >>>>> d. etc...?? > > > >>>>> > > > >>>>> The above implementation can be done in phases as follows: > > > >>>>> 1. Have webpage display application level information only > > > (read-only). > > > >>>>> Static display first (i.e. page needs to refresh to see latest > > > >>>>> data), next > > > >>>>> dynamically changing data. > > > >>>>> 2. Have jenkins build tools updated as needed by UI framework. > > > >>>>> 3. Update the backend (stram) to serve the UI pages > > > >>>>> 3. Extend the webage to display logical operator information. > > > >>>>> 4. Extend the webage to display physical operator information > > > >>>>> 5. Implement control operations on UI. > > > >>>>> > > > >>>>> > > > >>>>> To implement above functionality, I propose ReactJS as UI > framework > > > and > > > >>>>> MaterialUI (based on Google's material design) for > > theme/controls/CSS > > > >>>>> support. > > > >>>>> > > > >>>>> Please let me know what you think about the same. > > > >>>>> > > > >>>>> Also, please let know if anyone is interested in contributing to > > this > > > >>>>> feature. > > > >>>>> > > > >>>>> Thanks, > > > >>>>> Chinmay. > > > >>>>> > > > >> > > > > > > > > >
