[
https://issues.apache.org/jira/browse/SPARK-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165500#comment-14165500
]
Josh Rosen commented on SPARK-3644:
-----------------------------------
I think that a REST/JSON API is going to share many of the same design concerns
as a stable Java/Scala-based progress reporting API for Spark. It would be
great to use consistent naming across both APIs.
SPARK-2321 deals with a Java API to expose programmatic access to many of the
same things that a REST API would expose. I have a pull request open for
discussing the design of this API: https://github.com/apache/spark/pull/2696
It would be great if anyone here would comment on that PR / JIRA so that we can
work out the basic issues of what to expose / how to expose it in the Java API.
Once we've figured this out, providing a REST wrapper should be fairly trivial.
> REST API for Spark application info (jobs / stages / tasks / storage info)
> --------------------------------------------------------------------------
>
> Key: SPARK-3644
> URL: https://issues.apache.org/jira/browse/SPARK-3644
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Web UI
> Reporter: Josh Rosen
>
> This JIRA is a forum to draft a design proposal for a REST interface for
> accessing information about Spark applications, such as job / stage / task /
> storage status.
> There have been a number of proposals to serve JSON representations of the
> information displayed in Spark's web UI. Given that we might redesign the
> pages of the web UI (and possibly re-implement the UI as a client of a REST
> API), the API endpoints and their responses should be independent of what we
> choose to display on particular web UI pages / layouts.
> Let's start a discussion of what a good REST API would look like from
> first-principles. We can discuss what urls / endpoints expose access to
> data, how our JSON responses will be formatted, how fields will be named, how
> the API will be documented and tested, etc.
> Some links for inspiration:
> https://developer.github.com/v3/
> http://developer.netflix.com/docs/REST_API_Reference
> https://helloreverb.com/developers/swagger
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
