Re: Time-zone database issues

[Martin Buchholz]

Aligning the openjdk API with the tzdb format is a good thing to do long
term.

However, updating old jdk versions is sufficiently difficult (and some
people will want to keep jdk6 and jdk9 tzdata up to date for years) that
I'm already resigned to using rearguard format "forever". It's a simple
strategy that works well, even if we eventually need to make additional
modifications to the rearguarding script.

Maybe I should care more deeply about getting the exact hour of a timezone
transition in 1951 right?!

Re: Time-zone database issues

[Stephen Colebourne]

On Fri, 26 Oct 2018 at 08:52, Ramanand Patil  wrote:
> Hi Naoto,
> Thank you for filing the bug. As far as tzdata2018f release is concerned I am 
> going ahead with the integration, with the help of the patch provided here- 
> https://mm.icann.org/pipermail/tz/2018-October/027032.html Which avoids 25:00 
> in rearguard format.
> [ https://bugs.openjdk.java.net/browse/JDK-8213016 ]

Yes the right course of action for now is to use the patch that avoids 25:00.

Thanks Naoto for raising JDK-8212970 to cover the longer term change.

Stephen

RE: Time-zone database issues

[Ramanand Patil]

Hi Naoto,
Thank you for filing the bug. As far as tzdata2018f release is concerned I am 
going ahead with the integration, with the help of the patch provided here- 
https://mm.icann.org/pipermail/tz/2018-October/027032.html Which avoids 25:00 
in rearguard format.
[ https://bugs.openjdk.java.net/browse/JDK-8213016 ]


Regards,
Ramanand.

> -Original Message-
> From: Naoto Sato
> Sent: Thursday, October 25, 2018 8:48 PM
> To: Stephen Colebourne ; core-libs-dev  d...@openjdk.java.net>
> Subject: Re: Time-zone database issues
> 
> Thanks, Stephen.
> 
> I filed an issue for your suggestion:
> 
> https://bugs.openjdk.java.net/browse/JDK-8212970
> 
> I will need to look into the issue, but so far as I understand, will it be 
> fine to
> modify the offending transition date to the next day for 2018f's immediate
> issue?
> 
> Naoto
> 
> On 10/22/18 3:25 PM, Stephen Colebourne wrote:
> > The IANA time-zone database [1] provides details of how time-zones
> > change over time. The latest release - 2018f - cannot be processed
> > successfully by the current JDK parser [2].  A workaround exists
> > however unlike previous cases of tzdb incompatibility, this time it
> > makes sense for the JDK parser and API to be changed.
> >
> > Problem
> > ---
> > The JDK parser and API make the assumption that a time-zone change can
> > occur at any LocalTime or midnight-end-of-day, ie. from 00:00 to
> > 24:00. Unfortunately, the tzdb source files allow (and now include)
> > rules outside those valid values, in this case a value of 25:00.
> > Specifically, the rule that causes problems says that clocks change at
> > 25:00 on the first Saturday on or after the 8th of September.
> >
> > In the current problematic case, the rule can be rewritten to say that
> > clocks change at 01:00 on the first Sunday on or after the 9th of
> > September. However, there are cases where it is difficult to
> > impossible to rewrite the rule (such as 25:00 on the last Saturday in
> > a month, difficult because it goes into the next month).
> >
> > Proposed solution
> > 
> >
> > Fixing the parser to handle values like 25:00 would be relatively
> > easy. However, these rules are also exposed in the public API of
> > java.time.zone.ZoneOffsetTransitionRule [3]. Currently this class has
> > methods `getLocalTime()` and `isMidnightEndOfDay()`. These would need
> > to be deprecated and replaced by a new method `getLocalTimeDuration()`
> > (or some other name) that returns an instance of `Duration`.
> >
> > A user of ThreeTen-Backport [4] has provided a branch to do this, so I
> > know the change to be possible. However, since I have looked at the
> > code I cannot implement the change in OpenJDK (compromised IP). It
> > needs a cleanroom implementation by someone else.
> >
> > Is there agreement on the need for change? Is anyone (Oracle or
> > otherwise) willing to volunteer do the work?
> >
> > thanks
> > Stephen
> >
> >
> > [1] https://www.iana.org/time-zones
> > [2] https://bugs.openjdk.java.net/browse/JDK-8212684
> > [3]
> > https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time
> > /zone/ZoneOffsetTransitionRule.html
> > [4] https://www.threeten.org/threetenbp/
> >

Re: Time-zone database issues

[Naoto Sato]

Thanks, Stephen.

I filed an issue for your suggestion:

https://bugs.openjdk.java.net/browse/JDK-8212970

I will need to look into the issue, but so far as I understand, will it 
be fine to modify the offending transition date to the next day for 
2018f's immediate issue?


Naoto

On 10/22/18 3:25 PM, Stephen Colebourne wrote:

The IANA time-zone database [1] provides details of how time-zones
change over time. The latest release - 2018f - cannot be processed
successfully by the current JDK parser [2].  A workaround exists
however unlike previous cases of tzdb incompatibility, this time it
makes sense for the JDK parser and API to be changed.

Problem
---
The JDK parser and API make the assumption that a time-zone change can
occur at any LocalTime or midnight-end-of-day, ie. from 00:00 to
24:00. Unfortunately, the tzdb source files allow (and now include)
rules outside those valid values, in this case a value of 25:00.
Specifically, the rule that causes problems says that clocks change at
25:00 on the first Saturday on or after the 8th of September.

In the current problematic case, the rule can be rewritten to say that
clocks change at 01:00 on the first Sunday on or after the 9th of
September. However, there are cases where it is difficult to
impossible to rewrite the rule (such as 25:00 on the last Saturday in
a month, difficult because it goes into the next month).

Proposed solution


Fixing the parser to handle values like 25:00 would be relatively
easy. However, these rules are also exposed in the public API of
java.time.zone.ZoneOffsetTransitionRule [3]. Currently this class has
methods `getLocalTime()` and `isMidnightEndOfDay()`. These would need
to be deprecated and replaced by a new method `getLocalTimeDuration()`
(or some other name) that returns an instance of `Duration`.

A user of ThreeTen-Backport [4] has provided a branch to do this, so I
know the change to be possible. However, since I have looked at the
code I cannot implement the change in OpenJDK (compromised IP). It
needs a cleanroom implementation by someone else.

Is there agreement on the need for change? Is anyone (Oracle or
otherwise) willing to volunteer do the work?

thanks
Stephen


[1] https://www.iana.org/time-zones
[2] https://bugs.openjdk.java.net/browse/JDK-8212684
[3] 
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/zone/ZoneOffsetTransitionRule.html
[4] https://www.threeten.org/threetenbp/

Re: Time-zone database issues

[Stephen Colebourne]

On Thu, 25 Oct 2018 at 09:37, Andrew Haley  wrote:
>
> On 10/22/2018 11:25 PM, Stephen Colebourne wrote:
> > The IANA time-zone database [1] provides details of how time-zones
> > change over time. The latest release - 2018f - cannot be processed
> > successfully by the current JDK parser [2].  A workaround exists
> > however unlike previous cases of tzdb incompatibility, this time it
> > makes sense for the JDK parser and API to be changed.
>
> I don't think so. This patch is in the development version of tzdata:
>
> mm.icann.org/pipermail/tz/attachments/20181018/a1734c98/0001-Avoid-25-00-in-rearguard-format.patch

Perhaps I'm not making myself clear enough. We can cope with *this*
change either by using rearguard or changing the parser. However, the
tzdb author is determined to push ahead with whatever data changes he
sees fit (calls for more tzdb backwards compatibility are rejected).
At some point, he will add an out of bounds change here that can't be
represented in rearguard or in Java. We need to make a change here to
protect ourselves from that, and to fix what is in many ways a bug in
the Java API - caused by a misunderstanding of their file format.

Stephen

Re: Time-zone database issues

[Andrew Haley]

On 10/22/2018 11:25 PM, Stephen Colebourne wrote:
> The IANA time-zone database [1] provides details of how time-zones
> change over time. The latest release - 2018f - cannot be processed
> successfully by the current JDK parser [2].  A workaround exists
> however unlike previous cases of tzdb incompatibility, this time it
> makes sense for the JDK parser and API to be changed.

I don't think so. This patch is in the development version of tzdata:

mm.icann.org/pipermail/tz/attachments/20181018/a1734c98/0001-Avoid-25-00-in-rearguard-format.patch

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: Time-zone database issues

[Florian Weimer]

* Stephen Colebourne:

> On Tue, 23 Oct 2018 at 09:44, Florian Weimer  wrote:
>> * Stephen Colebourne:
>>
>> > No, the day-of-month and day-of-week would remain the same, as the
>> > time is relative to those dates.
>>
>> My expectation is that the values returned by the other methods would
>> change for a getLocalTime() that provides a normalized return value
>> because the transition rule in normalized time is different (i.e., last
>> Saturday in a month turns into something else).
>
> As mentioned in the original mail, it is not always possible to
> normalize the values back to sane values. Trying to do so would result
> in a worse outcome than the proposed change (more deprecations and
> more chance of users hitting compatibility problems).
>
> Current response (until tzdb made their recent change):
> - getDayOfWeek() = Sun
> - getLocalTime() = 01:00
>
> Proposed response:
> - getDayOfWeek() = Sat
> - getLocalTime() = 01:00 (deprecated)
> - getLocalTimeDuration() = 25:00

Hmm.  I'm no longer sure if normalization is always possible.  If the
rule says, “last Saturady in the month at 25:00”, there is no normalized
equivalent because every normalized rule can only express dates that
always fall into the same month, as far as I can see.

If this is correct, I think offering normalized and non-normalized
interfaces is futile.

Does it even make sense to expose this level of detail?  Many transition
rules cannot be expressed in terms of the Gregorian calendar anyway.

Thanks,
Florian

Re: Time-zone database issues

[Stephen Colebourne]

On Tue, 23 Oct 2018 at 09:44, Florian Weimer  wrote:
> * Stephen Colebourne:
>
> > No, the day-of-month and day-of-week would remain the same, as the
> > time is relative to those dates.
>
> My expectation is that the values returned by the other methods would
> change for a getLocalTime() that provides a normalized return value
> because the transition rule in normalized time is different (i.e., last
> Saturday in a month turns into something else).

As mentioned in the original mail, it is not always possible to
normalize the values back to sane values. Trying to do so would result
in a worse outcome than the proposed change (more deprecations and
more chance of users hitting compatibility problems).

Current response (until tzdb made their recent change):
- getDayOfWeek() = Sun
- getLocalTime() = 01:00

Proposed response:
- getDayOfWeek() = Sat
- getLocalTime() = 01:00 (deprecated)
- getLocalTimeDuration() = 25:00

Use of these methods is incredibly rare - it is a very specialist API
- and is supposed to match the data in tzdb. In addition, the most
important method of the class is createTransition(int) which is
unaffected by the changes. So the compatibility risk here really is
absolutely tiny.

Stephen

Re: Time-zone database issues

[Florian Weimer]

* Stephen Colebourne:

> No, the day-of-month and day-of-week would remain the same, as the
> time is relative to those dates.

My expectation is that the values returned by the other methods would
change for a getLocalTime() that provides a normalized return value
because the transition rule in normalized time is different (i.e., last
Saturday in a month turns into something else).

Thanks,
Florian

Re: Time-zone database issues

[Stephen Colebourne]

No, the day-of-month and day-of-week would remain the same, as the
time is relative to those dates.
I didn't provide that link for IP reasons! Please avoid clicking on
the link below!

Stephen


On Tue, 23 Oct 2018 at 06:45, Florian Weimer  wrote:
>
> * Stephen Colebourne:
>
> > Fixing the parser to handle values like 25:00 would be relatively
> > easy. However, these rules are also exposed in the public API of
> > java.time.zone.ZoneOffsetTransitionRule [3]. Currently this class has
> > methods `getLocalTime()` and `isMidnightEndOfDay()`. These would need
> > to be deprecated and replaced by a new method `getLocalTimeDuration()`
> > (or some other name) that returns an instance of `Duration`.
>
> I think getLocalTimeDuration() would have to be paired with new methods
> getDayOfMonthIndicator(), getDayOfWeek().  The values returned by these
> methods will have to change between normalized and non-normalized
> transition times.
>
> The branch appears to be here:
>
>   
>
> Thanks,
> Florian

Re: Time-zone database issues

[Florian Weimer]

* Stephen Colebourne:

> Fixing the parser to handle values like 25:00 would be relatively
> easy. However, these rules are also exposed in the public API of
> java.time.zone.ZoneOffsetTransitionRule [3]. Currently this class has
> methods `getLocalTime()` and `isMidnightEndOfDay()`. These would need
> to be deprecated and replaced by a new method `getLocalTimeDuration()`
> (or some other name) that returns an instance of `Duration`.

I think getLocalTimeDuration() would have to be paired with new methods
getDayOfMonthIndicator(), getDayOfWeek().  The values returned by these
methods will have to change between normalized and non-normalized
transition times.

The branch appears to be here:

  

Thanks,
Florian

Time-zone database issues

[Stephen Colebourne]

The IANA time-zone database [1] provides details of how time-zones
change over time. The latest release - 2018f - cannot be processed
successfully by the current JDK parser [2].  A workaround exists
however unlike previous cases of tzdb incompatibility, this time it
makes sense for the JDK parser and API to be changed.

Problem
---
The JDK parser and API make the assumption that a time-zone change can
occur at any LocalTime or midnight-end-of-day, ie. from 00:00 to
24:00. Unfortunately, the tzdb source files allow (and now include)
rules outside those valid values, in this case a value of 25:00.
Specifically, the rule that causes problems says that clocks change at
25:00 on the first Saturday on or after the 8th of September.

In the current problematic case, the rule can be rewritten to say that
clocks change at 01:00 on the first Sunday on or after the 9th of
September. However, there are cases where it is difficult to
impossible to rewrite the rule (such as 25:00 on the last Saturday in
a month, difficult because it goes into the next month).

Proposed solution


Fixing the parser to handle values like 25:00 would be relatively
easy. However, these rules are also exposed in the public API of
java.time.zone.ZoneOffsetTransitionRule [3]. Currently this class has
methods `getLocalTime()` and `isMidnightEndOfDay()`. These would need
to be deprecated and replaced by a new method `getLocalTimeDuration()`
(or some other name) that returns an instance of `Duration`.

A user of ThreeTen-Backport [4] has provided a branch to do this, so I
know the change to be possible. However, since I have looked at the
code I cannot implement the change in OpenJDK (compromised IP). It
needs a cleanroom implementation by someone else.

Is there agreement on the need for change? Is anyone (Oracle or
otherwise) willing to volunteer do the work?

thanks
Stephen


[1] https://www.iana.org/time-zones
[2] https://bugs.openjdk.java.net/browse/JDK-8212684
[3] 
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/zone/ZoneOffsetTransitionRule.html
[4] https://www.threeten.org/threetenbp/