[
https://issues.apache.org/jira/browse/SPARK-13241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154052#comment-15154052
]
Steve Loughran commented on SPARK-13241:
----------------------------------------
The REST API compatibility is described in
[monitoring|http://spark.apache.org/docs/latest/monitoring.html]. It explicitly
states
# Individual fields will never be removed for any given endpoint
# New fields may be added to existing endpoints
You can't change the existing field without breaking statement #1 and upsetting
people; but statement #2 provides the strategy for addressing this.
a {{long}} timestamp could have to go alongside the date. More of an issue than
the duplicate code is "what does a recipient do if they are different"; I'd go
for making the long normative, though even there there's the eternal ambiguity
over timezones, something probably ignored by now.
> add long--formatted timestamps to
> org.apache.spark.status.api.v1.ApplicationAttemptInfo
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-13241
> URL: https://issues.apache.org/jira/browse/SPARK-13241
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 1.6.0
> Reporter: Steve Loughran
>
> While writing tests to parse
> {{org.apache.spark.status.api.v1.ApplicationAttemptInfo}} coming off the
> history rest server, I can see that I can't actually unmarshall the
> timestamps without parsing the strings —and I fear that may be local specific.
> The problem here is that jersey is marshalling {{Date}} classes by calling
> {{Date.toString()}}, leaving an entry like
> {code}
> {
> "id": "application_1111_0000",
> "name": "spark-demo",
> "attempts": [
> {
> "attemptId": "1",
> "startTime": "2016-02-08T20:12:20.825GMT",
> "endTime": "1970-01-01T00:00:00.000GMT",
> "lastUpdated": "2016-02-08T20:12:20.848GMT",
> "duration": 0,
> "sparkUser": "hdfs",
> "completed": false
> }
> ]
> }
> {code}
> This is good for humans and the web UI, bad for any code trying to use the
> API for its own purposes.
> I propose adding, alongside the existing values, a simple {{long}} value for
> each time, representing the time in millis from the start of the epoch. This
> is trivial to marshall/unmarshall.
> This is easy to add, provided everyone is happy that adding a new field to
> the returned JSON is considered compatible with the existing REST API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
