Hi Community, I would like to propose a slightly different approach on having an UI for Apex.
The details are below: 1. This involves having an external web-server installed along side of apexCli. We will basically mimic all the functionalities that apexCli currently provides. 2. We will develop the necessary REST APIs for fetch/update operations which apexCli currently supports. 3. Because most of the information and operations needed by a typical user are already there in apexCli, it will of great help to end users and which may increase the adoption rate. About Chinmay's proposal of having the STRAM hosting the apps, please find my comments below: 1. It may increase load on STRAM if we were to provide real time display of information. I am aware that apexCli fetches info from STRAM apis but continuous polling can cause extra over-head. 2. The UI will have limited scope for the particular application. Users would definitely love to have a comprehensive UI for the management of Apex Apps as a whole. 3. UI might be able to affect the whole application through STRAM hence giving direct access to STRAM APIs might not be good idea. 4. Because STRAM can't have a comprehensive UI hosted inside, anyways we will have to develop an external one. So I would like propose the plan on action like this. 1. We develop the wrapper UI for apexCli in the first phase and give user all the management options as REST APIs. 2. If some of information we require are not in the current apexCli implementation, we can add those feature to apexCli and if required to STRAM APIs in subsequent phases. 3. In future we can even remove the REST server from STRAM if we could provide something like MetricsStore functionality which was there in DataTorrent RTS. -- Thanks and Regards, Shubhrajyoti Mohapatra On Tue, Jun 12, 2018 at 2:49 AM Vlad Rozov <vro...@apache.org> wrote: > IMO it will be hard to provide UI that meets everyone expectation. An > external webUI can serve two goals > > 1. provide reference UI implementation > 2. ensure that REST API have all necessary entry points > > by embedding webUI directly into stram it will be tempting to shortcut > some calls to use stram classes directly. > > Thank you, > > Vlad > > On 6/11/18 12:18, Pramod Immaneni wrote: > > I too think that a comprehensive UI works better standalone but given > that > > stram already runs a webapp server, provides web services and RM > provides a > > proxy with mapped url space for it, we can extend stram to provide better > > visual output. Overall I am +1 on the idea. > > > > On Mon, Jun 11, 2018 at 8:59 AM Vlad Rozov <vro...@apache.org> wrote: > > > >> Clarification: I am -0 on making UI part of the stram. I am +1 on > >> providing reference webUI application outside of the stram. > >> > >> Thank you, > >> > >> Vlad > >> > >> On 6/9/18 07:05, Vlad Rozov wrote: > >>> -0: stram provides REST API to query runtime information and IMO this > >>> is sufficient to build an external webUI application(s). > >>> > >>> Thank you, > >>> > >>> Vlad > >>> > >>> On 6/6/18 11:17, Ambarish Pande wrote: > >>>> +1 For this feature. It would be really helpfull for users to > >>>> visualize all > >>>> this information and the DAG. > >>>> > >>>> +1 for ReactJS + MaterialUI > >>>> A great choice, given its flexibility and performance. > >>>> > >>>> I would like to contribute to this effort. > >>>> > >>>> On Wed, 6 Jun 2018 at 8:13 PM, Chinmay Kolhatkar <chin...@apache.org> > >>>> wrote: > >>>> > >>>>> Hi Community, > >>>>> > >>>>> I want to propose a feature in apex-core to have a UI component in > >>>>> stram. > >>>>> This is influenced by how basic runtime information is shown on UI by > >>>>> spark. > >>>>> > >>>>> This includes following features: > >>>>> 1. Webpage will be hosted by stram and will be available for user to > >>>>> view > >>>>> in one of the two ways (I'll prefer approach b): > >>>>> a. Hosted on a static port in which case we'll need to add a new > >>>>> servlet > >>>>> to stram > >>>>> b. The webpage will be hosted on the same port as that of stram > >>>>> webservice but at a different path. Say, http:// > >>>>> <stram_host>:<webservice_port>/ui > >>>>> > >>>>> 2. The webpage will be a static page and depending on the framework > we > >>>>> chose, it can show the realtime metric data from stram. > >>>>> > >>>>> 3. There will be a categories of readonly information (maybe > >>>>> dynamically > >>>>> changing) that will be shown on the UI: > >>>>> a. Application level information > >>>>> i. Status > >>>>> ii. Number of tuples processed/emitted > >>>>> iii. Start time/ Running span > >>>>> iv. Stram events > >>>>> v. Total memory used/available > >>>>> vi. Number of container allocated/deployed > >>>>> v. Other information which is available in stram > >>>>> b. Logical Operator level information > >>>>> i. Status > >>>>> ii. Number of tuples processed/emitted > >>>>> iii. Important events related to logical operator > >>>>> iv. Container list in which the operator is deployed > >>>>> v. Any other information available in stram > >>>>> vi. Logical DAG View > >>>>> c. Container level information > >>>>> i. Status > >>>>> ii. Number of tuples processed/emitted > >>>>> iii. Important events related to logical operator > >>>>> iv. Any other information available in stram > >>>>> v. Physical DAG View > >>>>> > >>>>> 4. In second phase, there will be control related operations allowed > >>>>> from > >>>>> the UI, as follows: > >>>>> a. Stop/Kill application > >>>>> b. Stop/Kill containers > >>>>> c. Stack trace dump of containers > >>>>> d. Logs related? > >>>>> d. etc...?? > >>>>> > >>>>> The above implementation can be done in phases as follows: > >>>>> 1. Have webpage display application level information only > (read-only). > >>>>> Static display first (i.e. page needs to refresh to see latest > >>>>> data), next > >>>>> dynamically changing data. > >>>>> 2. Have jenkins build tools updated as needed by UI framework. > >>>>> 3. Update the backend (stram) to serve the UI pages > >>>>> 3. Extend the webage to display logical operator information. > >>>>> 4. Extend the webage to display physical operator information > >>>>> 5. Implement control operations on UI. > >>>>> > >>>>> > >>>>> To implement above functionality, I propose ReactJS as UI framework > and > >>>>> MaterialUI (based on Google's material design) for theme/controls/CSS > >>>>> support. > >>>>> > >>>>> Please let me know what you think about the same. > >>>>> > >>>>> Also, please let know if anyone is interested in contributing to this > >>>>> feature. > >>>>> > >>>>> Thanks, > >>>>> Chinmay. > >>>>> > >> > >
