If it is a set of static files that use existing REST API to display
information in a browser, where they will be hosted is irrelevant. They
can be hosted by stram, but it should be possible to host them
externally without making modificaitons. If this is the case, I don't
have any objections to the proposal.
Thank you,
Vlad
On 6/13/18 06:09, Chinmay Kolhatkar wrote:
Hi Team,
I agree with Thomas. The main focus of this proposal was to provide a UI
layer using existing REST APIs.
I think we should first focus on providing a meaningful UI served by AM to
user using the same existing REST APIs.
If in general, there is a agreement for the proposal, I can proceed with
creating a JIRA and feature branch for the same.
Thanks,
Chinmay.
On Wed, Jun 13, 2018 at 12:21 AM Thomas Weise <t...@apache.org> wrote:
Looks like there is some disconnect in this thread, a few replies express
concerns that the proposal IMO doesn't imply.
There is already a REST API that backs all of the CLI functionality. Modern
UIs run in the browser, and so what was proposed (unless I misunderstood)
would be based on the same REST API and nothing extra. The only additional
thing the AM has to do is serve static files. And AFAIK it would actually
simplify the UI implementation when files and REST API have the same
origin.
To me where files are coming from is actually a smaller piece and something
that can be augmented. But before proposing solutions outside of Apex or
additional deployment requirements, please consider the user experience. It
should be really simple to setup and use the UI.
I would expect that a large part of the discussion evolves around the UI
functionality that helps the target persona.
Thanks,
Thomas
On Tue, Jun 12, 2018, 8:05 PM Shubhrajyoti Mohapatra <
sjmohapat...@gmail.com>
wrote:
Hi Community,
I would like to propose a slightly different approach on having an UI for
Apex.
The details are below:
1. This involves having an external web-server installed along side of
apexCli. We will basically mimic all the functionalities that apexCli
currently provides.
2. We will develop the necessary REST APIs for fetch/update operations
which apexCli currently supports.
3. Because most of the information and operations needed by a typical
user are already there in apexCli, it will of great help to end users
and
which may increase the adoption rate.
About Chinmay's proposal of having the STRAM hosting the apps, please
find
my comments below:
1. It may increase load on STRAM if we were to provide real time
display
of information. I am aware that apexCli fetches info from STRAM apis
but
continuous polling can cause extra over-head.
2. The UI will have limited scope for the particular application.
Users
would definitely love to have a comprehensive UI for the management of
Apex
Apps as a whole.
3. UI might be able to affect the whole application through STRAM
hence
giving direct access to STRAM APIs might not be good idea.
4. Because STRAM can't have a comprehensive UI hosted inside, anyways
we
will have to develop an external one.
So I would like propose the plan on action like this.
1. We develop the wrapper UI for apexCli in the first phase and give
user all the management options as REST APIs.
2. If some of information we require are not in the current apexCli
implementation, we can add those feature to apexCli and if required to
STRAM APIs in subsequent phases.
3. In future we can even remove the REST server from STRAM if we could
provide something like MetricsStore functionality which was there in
DataTorrent RTS.
--
Thanks and Regards,
Shubhrajyoti Mohapatra
On Tue, Jun 12, 2018 at 2:49 AM Vlad Rozov <vro...@apache.org> wrote:
IMO it will be hard to provide UI that meets everyone expectation. An
external webUI can serve two goals
1. provide reference UI implementation
2. ensure that REST API have all necessary entry points
by embedding webUI directly into stram it will be tempting to shortcut
some calls to use stram classes directly.
Thank you,
Vlad
On 6/11/18 12:18, Pramod Immaneni wrote:
I too think that a comprehensive UI works better standalone but given
that
stram already runs a webapp server, provides web services and RM
provides a
proxy with mapped url space for it, we can extend stram to provide
better
visual output. Overall I am +1 on the idea.
On Mon, Jun 11, 2018 at 8:59 AM Vlad Rozov <vro...@apache.org>
wrote:
Clarification: I am -0 on making UI part of the stram. I am +1 on
providing reference webUI application outside of the stram.
Thank you,
Vlad
On 6/9/18 07:05, Vlad Rozov wrote:
-0: stram provides REST API to query runtime information and IMO
this
is sufficient to build an external webUI application(s).
Thank you,
Vlad
On 6/6/18 11:17, Ambarish Pande wrote:
+1 For this feature. It would be really helpfull for users to
visualize all
this information and the DAG.
+1 for ReactJS + MaterialUI
A great choice, given its flexibility and performance.
I would like to contribute to this effort.
On Wed, 6 Jun 2018 at 8:13 PM, Chinmay Kolhatkar <
chin...@apache.org>
wrote:
Hi Community,
I want to propose a feature in apex-core to have a UI component
in
stram.
This is influenced by how basic runtime information is shown on
UI
by
spark.
This includes following features:
1. Webpage will be hosted by stram and will be available for user
to
view
in one of the two ways (I'll prefer approach b):
a. Hosted on a static port in which case we'll need to add a
new
servlet
to stram
b. The webpage will be hosted on the same port as that of
stram
webservice but at a different path. Say, http://
<stram_host>:<webservice_port>/ui
2. The webpage will be a static page and depending on the
framework
we
chose, it can show the realtime metric data from stram.
3. There will be a categories of readonly information (maybe
dynamically
changing) that will be shown on the UI:
a. Application level information
i. Status
ii. Number of tuples processed/emitted
iii. Start time/ Running span
iv. Stram events
v. Total memory used/available
vi. Number of container allocated/deployed
v. Other information which is available in stram
b. Logical Operator level information
i. Status
ii. Number of tuples processed/emitted
iii. Important events related to logical operator
iv. Container list in which the operator is deployed
v. Any other information available in stram
vi. Logical DAG View
c. Container level information
i. Status
ii. Number of tuples processed/emitted
iii. Important events related to logical operator
iv. Any other information available in stram
v. Physical DAG View
4. In second phase, there will be control related operations
allowed
from
the UI, as follows:
a. Stop/Kill application
b. Stop/Kill containers
c. Stack trace dump of containers
d. Logs related?
d. etc...??
The above implementation can be done in phases as follows:
1. Have webpage display application level information only
(read-only).
Static display first (i.e. page needs to refresh to see latest
data), next
dynamically changing data.
2. Have jenkins build tools updated as needed by UI framework.
3. Update the backend (stram) to serve the UI pages
3. Extend the webage to display logical operator information.
4. Extend the webage to display physical operator information
5. Implement control operations on UI.
To implement above functionality, I propose ReactJS as UI
framework
and
MaterialUI (based on Google's material design) for
theme/controls/CSS
support.
Please let me know what you think about the same.
Also, please let know if anyone is interested in contributing to
this
feature.
Thanks,
Chinmay.
