Github user Aalnafessah commented on the issue:
https://github.com/apache/spark/pull/15803
Hi All
I have an issue with the Spark history time zone
When I submit a job on a set of machines machine located in London
timezone, the Spark Master on the dashboard has the correct
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
ok ,thanks~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/15803
@windpiger given the related JIRA was closed as duplicate to a fixed issue
could you close this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
It is slways showing UTC in the main page, but with server timezone in
other pages like last page I've pasted. Not sure if it's true. Any way we'll
waiting for the results from @windpiger ,
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
I don't think that's the conclusion here @windpiger -- it would however be
nice to correctly show the time zone in all UIs, whatever it is currently.
---
If your project is set up for it, you can
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
I will check other UIs ,and fix them to the UTC
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
To be clear, the history server already shows UTC always right? this change
depends on that being true.
Yes, I think it's not true of the other UIs. We already have this
inconsistency.
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
It's always better to show a timezone in table header, i think, no matter
what the timezone it really uses.
But changing to show GMT/UTC always? I have to say it's a bold move even
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
OK, I suppose there is some logic to showing GMT/UTC always in the history
server; it shows a history over jobs from many drivers and so doesn't as much
make sense to be specific to the timezone of
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
I would like solution 1, as other time string in Spark UI shows, like
JobPage:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
there are three things one could do with a date:
1. Display in server-side timezone
2. Display in client-side timezone
3. Display in UTC/GMT always
The original change went
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
we need the timezone of the server side, or we just use epoch time to
converted to GMT readable time(then show the timezone of the serverside
dependent on the page which is configured by the
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
Do we have a guy who's good at JAX-RS? maybe he can explain the theory and
help us to understand better :)
---
If your project is set up for it, you can reply to this email and have your
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
@srowen Before the code changes, browser get date string from server side,
now instead it get Date(this conclusion comes from codes
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
We are talking about the history server's human-readable API, right? the
code being changed does not deal with JSON Dates, but hacks a date string to
drop seconds and timezone.
My point was
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
@ajbozarth In browser side the timezone used to build Date from epoch time
is the one **at browser side**, not that one in **History Server side**. These
two are different in many cases. So
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/15803
The json also contains the epoch time as an int, can't we use that to build
a local time in the browser, or is that harder than I imagine?
---
If your project is set up for it, you can reply to
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
I think the problem is that Date tranfered in REST ways take no timezone,
one possible reason is :
http://stackoverflow.com/questions/23730062/use-iso-8601-dates-in-jax-rs-responses.
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
Like what showed in image, user can get app infos in two ways:
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
I'll post how UI works and what changes it did to be different before later
:)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
Hm, so I looked at how the other UIs work, and they seem to not be in GMT
always. They happen to use the machines' default time zone by way of using a
`SimpleDateFormat` to render times. So it has
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/15803
ah, ok I did misunderstand, I thought you were talking changing timezone
based on browser local time.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
I tend to agree with the view that all times should be UTC on the server
side, and perhaps even client side for consistency. This is maybe a separate
discussion though.
---
If your project is set
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
Nah i think we have a misunderstand here. @tgravescs
If understand right, what you mean is that most companies run their server
in UTC timezone. That's OK. Under that condition,
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
Right now, the times are already all in GMT. It would be at least less
change to leave it that way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/15803
The timezone is going to vary by setup. Normally I would expect any large
company that covers more then one timezone to run machines in UTC to
consistency and auditing purposes since they could
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
I agree with showing the timezone with date string.
But always using GMT/UTC time is not a good choice, logs of
application(using log4j) usually are printed using local
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
@srowen I have modify it by put GMT on the column name to tell user the
time showed in the list is GMT time.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/15803
Have you looked into using a JS Date object? I don't have much experience
with it but it may solve this issue
---
If your project is set up for it, you can reply to this email and have your
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
This makes it harder to read though, because it undoes the slightly hacky
'parsing' of the ISO 8601 string to make it a bit more readable, taking out the
partial seconds and timezone and 'T'. What
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
@srowen @tgravescs I aggree with you that, we should explicit show the
timezone, so I modify the pr simply that it show the GMT time, such as the
picture in the pr description.
But I
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/15803
I agree with @srowen we should specify the time zone. Personally I prefer
this to stay UTC because its more standard and because that way the times on UI
can be easily used to correlate times
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
Aha,I got it ~ Thanks~
Add the timezone info after the time showed to the user, is it ok?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
@srowen If show the same GMT time, I think the user maybe more ambiguous,
or the user should translate it to their local time in their mind when they see
the GMT, which is not nicely.
---
If
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
I'm not disagreeing with that. I am saying that whatever the timezone is,
it should be displayed. That's not reflected in this change right now.
---
If your project is set up for it, you can reply
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/15803
(FWIW, this reminds me of
[SPARK-18350](https://issues.apache.org/jira/browse/SPARK-18350))
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
(Timestamps don't even have a timezone.) Of course, time should be stored
as a timestamp. I am saying something else, that when rendered as
human-readable time, it needs to render a timezone
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
@srowen UTC timestamp(long) is the standard time, no matter it is
translated to which timezone, the long value is the same. But the readable date
string should be nicely adapter to the user's
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15803
General comment: we should always render times with a timezone for clarity,
even if it's in a column header. This has been the source of so many problems
I've seen over the years.
---
If your
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
@ajbozarth could you help to review this? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user windpiger commented on the issue:
https://github.com/apache/spark/pull/15803
UTC timestamp can translate to different zone's local time,which the user
see from the history ui.
It is reasonable to return UTC/GMT time to the user browser, and show it by
translate to the
Github user WangTaoTheTonic commented on the issue:
https://github.com/apache/spark/pull/15803
thanks for the fix.
This patch parse the timestamp instead of the Date String returned. The
REST api still return the GMT time, which is insistent with UI showing.
I've
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/15803
If I remember correctly, does not UI change require a screenshot in the PR
description?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15803
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
44 matches
Mail list logo
