+1 for a UI for Apex. -0 for making it part of the STRAM.
- Regarding same port, I was wondering if there will be usage patterns based on firewall rules just based on ports ? This leads us to a situation where we have to trust all users at the same level as the application communication patterns. - Deepak N mentioned of building metrics components. Assuming he is working on this feature and is providing a REST API as well, there might be overlaps in the functionality being implemented. Some part of the UI features mentioned in the list seem to be definitely metrics related. Most of the metrics UI views can be aggregated at external web applications like Grafana of course with the help of other tooling like REST server from atrato or Prometheus components. - Are we considering serving metrics beyond application restarts? If yes, visualising metrics is a better fit for tooling carved out for those purposes given the amount of information will be huge and we will also need potentially time series stores ? - The highest value is from the points mentioned in the second phase and DAG views implementation in the first phase.( assuming Deepak N proposal for metrics is implementing a REST API). There is definitely a huge value add in providing a UI and the need of the hour for Apex. My concern is that we might be diluting the value add provided to build such a functionality inside the STRAM and hence feel it is better served as an external application as opposed to something embedded in the STRAM. Regards, Ananth On Sun, Jun 10, 2018 at 3:10 AM, Thomas Weise <t...@apache.org> wrote: > strong +1 for adding a UI to Apex. This is actually something that users > expect and are accustomed to with similar projects. Thanks for taking the > initiative. > > There are nuances to be discussed, but the overall plan LGTM. > > To your question, I think this should use the existing web port with a > different path, this will make it easy to expose through the RM proxy. The > question whether the port is static or dynamic is a different concern, > there could be a separate feature that optionally allows the user to assign > a static port. On YARN, there is already a static address, the RM proxy > port. > > I would propose to start this feature on a branch. The requirements for > build integration and source code structure should be fleshed out as you > go. It is conceivable that the UI build, which requires separate tooling, > runs as part of the release build, but not the usual CI. > > Thomas > > > > On Wed, Jun 6, 2018 at 7:43 AM, Chinmay Kolhatkar <chin...@apache.org> > wrote: > > > Hi Community, > > > > I want to propose a feature in apex-core to have a UI component in stram. > > This is influenced by how basic runtime information is shown on UI by > > spark. > > > > This includes following features: > > 1. Webpage will be hosted by stram and will be available for user to view > > in one of the two ways (I'll prefer approach b): > > a. Hosted on a static port in which case we'll need to add a new > servlet > > to stram > > b. The webpage will be hosted on the same port as that of stram > > webservice but at a different path. Say, http:// > > <stram_host>:<webservice_port>/ui > > > > 2. The webpage will be a static page and depending on the framework we > > chose, it can show the realtime metric data from stram. > > > > 3. There will be a categories of readonly information (maybe dynamically > > changing) that will be shown on the UI: > > a. Application level information > > i. Status > > ii. Number of tuples processed/emitted > > iii. Start time/ Running span > > iv. Stram events > > v. Total memory used/available > > vi. Number of container allocated/deployed > > v. Other information which is available in stram > > b. Logical Operator level information > > i. Status > > ii. Number of tuples processed/emitted > > iii. Important events related to logical operator > > iv. Container list in which the operator is deployed > > v. Any other information available in stram > > vi. Logical DAG View > > c. Container level information > > i. Status > > ii. Number of tuples processed/emitted > > iii. Important events related to logical operator > > iv. Any other information available in stram > > v. Physical DAG View > > > > 4. In second phase, there will be control related operations allowed from > > the UI, as follows: > > a. Stop/Kill application > > b. Stop/Kill containers > > c. Stack trace dump of containers > > d. Logs related? > > d. etc...?? > > > > The above implementation can be done in phases as follows: > > 1. Have webpage display application level information only (read-only). > > Static display first (i.e. page needs to refresh to see latest data), > next > > dynamically changing data. > > 2. Have jenkins build tools updated as needed by UI framework. > > 3. Update the backend (stram) to serve the UI pages > > 3. Extend the webage to display logical operator information. > > 4. Extend the webage to display physical operator information > > 5. Implement control operations on UI. > > > > > > To implement above functionality, I propose ReactJS as UI framework and > > MaterialUI (based on Google's material design) for theme/controls/CSS > > support. > > > > Please let me know what you think about the same. > > > > Also, please let know if anyone is interested in contributing to this > > feature. > > > > Thanks, > > Chinmay. > > >
