Re: RFR(s): 4285505: deprecate java.lang.Compiler

[Tim Ellison]

On 08/09/16 17:08, Krystal Mok wrote:
> Thanks for your reply. It's good to know at least the known use cases
> from IBM have indeed been discussed.
> 
> On Thu, Sep 8, 2016 at 9:01 AM, Tim Ellison <t.p.elli...@gmail.com
> <mailto:t.p.elli...@gmail.com>> wrote:
> 
> On 07/09/16 23:45, Krystal Mok wrote:
> > I see that on the JBS page, your most recent comment says it's been 
> decided
> > that for JDK9 it's okay to deprecate and forRemoval=true, while also
> > mentioning the uses of this class in IBM's implementation.
> >
> > Does that mean IBM has agreed on the deprecation of this class?
> 
> Yep, I've seen those references and am ok with the deprecation in JDK9.
> 
> > I thought J9 had features that allowed Java applications to do
> > fine-grained control over the JIT compiler at runtime, e.g. force
> > compilation of specified methods *at some certain point* in the
> > program.
> 
> Not sure what you are thinking of there Kris.  We do implement the five
> public methods on Compiler to do pretty much what you would expect:
> 
> java.lang.Compiler.command(Object any) - this only does something for a
> couple of custom arguments, most objects passed to it are simply
> ignored.
> 
> java.lang.Compiler.compileClass(Class clazz) - the JIT will compile
> all methods in the given class. These compilations are synchronous, i.e.
> the application thread that called the API will wait until all
> compilations are finished.
> 
> This and the next one are the ones I was talking about: these calls can
> be made at arbitrary points in time during Java program execution, so
> they are "dynamic".
> For instance, I may have a program that wishes to convey phase shift
> properties to the VM, by calling once of these methods right before the
> phase shift happens. That's something a static configuration means (e.g.
> compiler configuration file) won't be able to do. Ditto for the
> "waitOnCompilationQueue" command.

Yes, that is one way in which we can have an application indicate to the
JIT that it is moving from start-up to run phase, and we have played
with that.  But come JDK9 and beyond there are much better opportunities
for start-up optimizations that means these APIs can be removed without
grief.

Regards,
Tim

> java.lang.Compiler.compileClasses(String s) - the JIT will compile all
> methods from classes that match the given string.
> 
> java.lang.Compiler.disable() - disables all future JIT compilations.
> 
> java.lang.Compiler.enable() - resumes JIT compilations.
> 
> IMO dropping these APIs would not be a great loss.
> 
> > What JEP 165 is proposing can only statically configure JIT behaviors 
> for
> > HotSpot. The same approach doesn't seem to cover what J9 used to 
> support.
> >
> > Could you please share more background on the discussions that led to 
> the
> > decision?
> 
> As you would expect, IBM and Oracle talk regularly about all things
> Java, and this topic was raised as a heads-up that it was coming to
> OpenJDK.  So there really isn't any background to the discussion.
> 
> Regards,
> Tim
> (IBM Java team)
> 
> > On Wed, Sep 7, 2016 at 2:50 PM, Stuart Marks
> <stuart.ma...@oracle.com <mailto:stuart.ma...@oracle.com>>
> > wrote:
> >
> >> Tomorrow's headline: Oracle Proposes To Remove Java JIT Compiler
> >>
> >> :-)
> >>
> >>
> >> On 9/7/16 2:44 PM, Remi Forax wrote:
> >>
> >>> Yes, i like this patch :)
> >>>
> >>> Aleksey, It's worst than what you think because for a lot of people
> >>> Compiler == java compiler and not JIT compiler so they try to
> compile a
> >>> .java file with the method compileClasses(String).
> >>>
> >>> I'm glad this class will disappear soon.
> >>>
> >>> Rémi
> >>>
> >>> - Mail original -
> >>>
> >>>> De: "Aleksey Shipilev" <aship...@redhat.com
> <mailto:aship...@redhat.com>>
> >>>> À: "Stuart Marks" <stuart.ma...@oracle.com
> <mailto:stuart.ma...@oracle.com>>, "core-libs-dev" <
> >>>> core-libs-dev@openjdk.java.net
> <mailto:core-libs-dev@openjdk.java.net>>
> >>>> Envoyé: Mercredi 7 Septembre 2016 23:34:02
> >>>> Objet: Re: RFR(s): 4285505: deprecate java.lang.Compiler
> >>>>
> >>>
> >>> On 09/07/2016 11:52 PM, Stuart Marks wrote:
> >>>>
> >>>>> Please review this small patch to deprecate java.lang.Compiler for
> >>>>> removal.
> >>>>>
> >>>>
> >>>> Yes, +1.
> >>>> This class is very confusing to have these days.
> >>>>
> >>>> -Aleksey
> >>>>
> >>>
> 
> 

Re: RFR(s): 4285505: deprecate java.lang.Compiler

[Tim Ellison]

On 07/09/16 23:45, Krystal Mok wrote:
> I see that on the JBS page, your most recent comment says it's been decided
> that for JDK9 it's okay to deprecate and forRemoval=true, while also
> mentioning the uses of this class in IBM's implementation.
> 
> Does that mean IBM has agreed on the deprecation of this class?

Yep, I've seen those references and am ok with the deprecation in JDK9.

> I thought J9 had features that allowed Java applications to do 
> fine-grained control over the JIT compiler at runtime, e.g. force
> compilation of specified methods *at some certain point* in the
> program.

Not sure what you are thinking of there Kris.  We do implement the five
public methods on Compiler to do pretty much what you would expect:

java.lang.Compiler.command(Object any) - this only does something for a
couple of custom arguments, most objects passed to it are simply ignored.

java.lang.Compiler.compileClass(Class clazz) - the JIT will compile
all methods in the given class. These compilations are synchronous, i.e.
the application thread that called the API will wait until all
compilations are finished.

java.lang.Compiler.compileClasses(String s) - the JIT will compile all
methods from classes that match the given string.

java.lang.Compiler.disable() - disables all future JIT compilations.

java.lang.Compiler.enable() - resumes JIT compilations.

IMO dropping these APIs would not be a great loss.

> What JEP 165 is proposing can only statically configure JIT behaviors for
> HotSpot. The same approach doesn't seem to cover what J9 used to support.
> 
> Could you please share more background on the discussions that led to the
> decision?

As you would expect, IBM and Oracle talk regularly about all things
Java, and this topic was raised as a heads-up that it was coming to
OpenJDK.  So there really isn't any background to the discussion.

Regards,
Tim
(IBM Java team)

> On Wed, Sep 7, 2016 at 2:50 PM, Stuart Marks 
> wrote:
> 
>> Tomorrow's headline: Oracle Proposes To Remove Java JIT Compiler
>>
>> :-)
>>
>>
>> On 9/7/16 2:44 PM, Remi Forax wrote:
>>
>>> Yes, i like this patch :)
>>>
>>> Aleksey, It's worst than what you think because for a lot of people
>>> Compiler == java compiler and not JIT compiler so they try to compile a
>>> .java file with the method compileClasses(String).
>>>
>>> I'm glad this class will disappear soon.
>>>
>>> Rémi
>>>
>>> - Mail original -
>>>
 De: "Aleksey Shipilev" 
 À: "Stuart Marks" , "core-libs-dev" <
 core-libs-dev@openjdk.java.net>
 Envoyé: Mercredi 7 Septembre 2016 23:34:02
 Objet: Re: RFR(s): 4285505: deprecate java.lang.Compiler

>>>
>>> On 09/07/2016 11:52 PM, Stuart Marks wrote:

> Please review this small patch to deprecate java.lang.Compiler for
> removal.
>

 Yes, +1.
 This class is very confusing to have these days.

 -Aleksey

>>>

Re: JEP 254: Compact Strings - length limits

[Tim Ellison]

On 06/09/16 19:04, Xueming Shen wrote:
> On 9/6/16, 10:09 AM, Tim Ellison wrote:
>> Has it been noted that while JEP 254 reduces the space occupied by one
>> byte per character strings, moving from a char[] to byte[]
>> representation universally means that the maximum length of a UTF-16
>> (two bytes per char) string is now halved?

Hey Sherman,

> Yes, it's a known "limit" given the nature of the approach. It is
> not considered to be an "incompatible change", because the max length
> the String class and the corresponding buffer/builder classes can
> support is really an implementation details, not a spec requirement.

Don't confuse spec compliance with compatibility.  Of course, the JEP
should not break the formal specified behavior of String etc, but the
goal was to ensure that the implementation be compatible with prior
behavior. As you know, there are many places where compatible behavior
beyond the spec is important to maintain.

> The conclusion from the discussion back then was this is something we
> can trade off for the benefits we gain from the approach. 

Out of curiosity, where was that?  I did search for previous discussion
of this topic but didn't see it -- it may be just my poor search foo.

> Do we have a real use case that impacted by this change?

People stash all sorts of things in (immutable) Strings. Reducing the
limits in JDK9 seems like a regression.  Was there any consideration to
using the older Java 8 StringCoding APIs for UTF-16 strings (already
highly perf tuned) and adding additional methods for compact strings
rather than rewriting everything as byte[]'s?

Regards,
Tim

>> Since the goal is "preserving full compatibility", this has been missed
>> by failing to allow for UTF-16 strings of length greater than
>> Integer.MAX_VALUE / 2.
>>
>> Regards,
>> Tim
>>
>>
> 

Re: j.u.regex: Negated Character Classes

[Tim Ellison]

Hi Sherman, ok so I'll admit to reading through to the end of your note
and finding it interesting ;-)

Some comments in-lined.

On 03/Jun/2011 22:55, Xueming Shen wrote:
 I'm sure everybody understands what negated character classes [^...]
 in j.u.regex means.
 You would never have doubt about
 
 [^c] does NOT match c
 [^0-9] does NOT match 8
 [^a-z] does NOT match b
 [^a-bc-d] does NOT match 'c
 
 But how about
 
 does [^[c]] match c?
 does [^[0-9]] match 8?
 does [^[a-z]] match b?
 does [^a-b[c-d]] match c?
 
 I was wrong on all of them when was asked first time and it took me
 a while to figure out what is going on behind it. Oh, btw, the answer
 is yes for all 4, yes, the
 
 [^[c]] matches c
 [^[0-9]] matches 8
 [^[a-z]] matches b.
 [^a-b[c-d]] matches c  (while [^a-bc-d] does NOT match c)

I would not have known the right answer to this quiz either; it seems
that the use of nested character sets is sufficiently rare that we've
not had to learn how these behave.

 Another interesting sample is
 
 [^a-b[c-d]e-f] matches c but does NOT match e (so the e-f part after
 the nested character class [c-d] does back to normal).
 
 It appears the negation of the outer character class does not
 affect its nested character class,

I think the easiest way to explain the situation is not to consider the
negation separately, but that [^ is the syntax of a negated character
set.  Having a normal character set inside a negated character set then
seems ok to me.

 so [^X] is always opposite from
 [^[X]], X to be any character class.
 
 Same strange thing seems to be true for intersection operation 
 as well, so both [a-dc-f] and [^a-dc-f] do NOT match a.
 
 This does not sound correct, at least for me.

This case is, I agree, a gotcha which is hard to justify through the
syntax rather than the implementation.

 The source code suggests that we are treating the nested/embedded
 [...] character class and the intersection  specially, so
 
 [^[X]  is interpreted as [^] union [X]
 [^X[Y]] is interpreted as [^X] union [Y]
 [^X[Y]Z] is interpreted as [^XZ] union [Y]
 [^XY] is interpreted as [^X]  Y
 
 What I meant treating...specially is that we do NOT do the same
 thing for other embedded character classes, so while [^[a-z]] does
 match c, [^\p{Lower}] actually does NOT match c, which I would
 expect.

 The j.u.regex.Pattern APIs do NOT help. All the samples given for 
 Character classes[1] section are simple negation, no nested
 sample is given. And neither ^ nor [^...] appear in the operator
 precedence table. The behaviors in other regex engines, such as Perl
 and POSIX, don't help, as nested character class is not supported
 there.
 
 I did check with the original author who wrote this part of the code.
 It appears the current implementation is indeed what he intended to
 do back then, so this behavior is NOT an implementation bug but by
 design.
 
 Personally I don't feel this design is not correct.

More mind tricks ;-) ?  You think the design *is* wrong?

 Ideally, I would assume the spec either specifies [^...] as a
 separate group operator to be the complement of [...], or ^ as
 the negation operator with the lowest precedence, such as (from
 lowest to highest)
 
 (1) Negation  ^(only at the beginning of the [...])
 (2) Intersection 
 (3) Range -
 (4) nested class []

or as I suggested above
  (5) negated class [^ ... ]

and the understanding that nested classes do not 'inherit' the negation
property of their parent.

 
 So
 [^X[Y]] would be the complement of [Xunion[Y]]
 
 [^X[Y]Z] would be the complement of  [Xunion[Y]unionZ]
 
 [^XY] would be the complement of [XY]
 
 for example, if I dump the regex internal logic node tree for the sample
 regex
 [^a-b[c-d]e-f], the jdk7 and jdk8 results would look like
 
 /home/sherman/TL/regex$
 /export/sherman/Workspace/jdk7/jdk/build/linux-i586/bin/java RegEx -flag
 1000 [^a-b[c-d]e-f] c
 Pattern=[^a-b[c-d]e-f]
 Input  =c
  1: Difference (7)
  2: Union (0)
  3: Complement (0)
  4: Range [a-b] (0)
  5: Range [c-d] (0)
  6: Range [e-f] (0)
  7: END (0)
 ---
 match:true
 groupCount=0
 
 /home/sherman/TL/regex$
 /export/sherman/Workspace/jdk8/jdk/build/linux-i586/bin/java RegEx -flag
 1000 [^a-b[c-d]e-f] c
 Pattern=[^a-b[c-d]e-f]
 Input  =c
  1: Complement (7)
  2: Union (0)
  3: Union (0)
  4: Range [a-b] (0)
  5: Range [c-d] (0)
  6: Range [e-f] (0)
  7: END (0)
 ---
 match:false
 
 I know, most of people might not be interested, but if you have read 
 this far, means you are interested:-) and might have some opinions.
 It is definitely an incompatible change, given this has been the
 behavior from the very beginning of Java regex and has been there for
 almost a decade, I would appreciate any comment/opinion, especially
 if you agree that the existing behavior is NOT correct and therefor
 we need to fix, OR you think the existing one is just