[
https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14192502#comment-14192502
]
Marcelo Vanzin commented on SPARK-1537:
---------------------------------------
bq. This is proposed to improve the Java libs by adding GET APIs. They are used
to query data, NOT to put data.
Spark needs both to put and read data, otherwise the ATS is useless for Spark.
The current goal of Spark is to use the ATS as a store for its history data,
since the data itself is not considered public and stable itself.
So there is no point in integration if you can only write data. (I know you can
read data through other means, but I don't want to write a custom REST client
just to get ATS support in.)
bq. It is reported for 2.5, and is probably no longer valid after we fixed a
bunch of security issues for 2.6.
I'm not sure why you say it's security-related since there nothing
security-related in the example code I posted. And if something doesn't work in
2.5 but works in 2.6, it means we (and by that I mean Spark) have to restrict
our support to the versions where things work - even if the underlying API is
exactly the same.
> Add integration with Yarn's Application Timeline Server
> -------------------------------------------------------
>
> Key: SPARK-1537
> URL: https://issues.apache.org/jira/browse/SPARK-1537
> Project: Spark
> Issue Type: New Feature
> Components: YARN
> Reporter: Marcelo Vanzin
> Assignee: Marcelo Vanzin
>
> It would be nice to have Spark integrate with Yarn's Application Timeline
> Server (see YARN-321, YARN-1530). This would allow users running Spark on
> Yarn to have a single place to go for all their history needs, and avoid
> having to manage a separate service (Spark's built-in server).
> At the moment, there's a working version of the ATS in the Hadoop 2.4 branch,
> although there is still some ongoing work. But the basics are there, and I
> wouldn't expect them to change (much) at this point.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
