Re: RFC: JEP JDK-8208089: Implement C++14 Language Features
Kim, Thank you for posting this. It's an important step to make. I wanted to clarify a couple of points: 1. Since this is infrastructure JEP, is the version of JDK which will undergo such changes going to be some future version or does it apply to past versions as well? I'm concerned with how we can simplify security backports to 8u which (I currently assume) is not subject to this change. 2. Which versions of GCC do you tentatively consider at this point? Non-x86 ports may have a problem upgrading to a specific version of GCC which the shared code will use and may need additional lead time to adjust. Thanks, -Aleksei On 03/10/2018 22:13, Kim Barrett wrote: I've submitted a JEP for (1) enabling the use of C++14 Language Features when building the JDK, (2) define a process for deciding and documenting which new features can be used or are forbidden in HotSpot code, (3) provide an initial list of permitted and forbidden new features. https://bugs.openjdk.java.net/browse/JDK-8208089
Re: Review Request: 6202130: java.util.jar.Attributes.writeMain() can't handle multi-byte chars
"utf8" is a thing. "utf" is not.
On Thu, Oct 4, 2018 at 12:58 PM, Philipp Kunz wrote:
> Thanks to Sherman's hint, I revised the test to use the terms unicode
> character and utf encoding consistently and not utf character. Affects
> only the test, mostly in comments and a few identifiers.
>
> Philipp
>
>
> On Wed, 2018-10-03 at 12:48 -0700, Xueming Shen wrote:
>> +1
>>
>> only nit is it might be preferred to use "utf8" directly in the test
>> instead of current "utf"
>>
>> -sherman
>>
>> On 10/03/2018 11:55 AM, Philipp Kunz wrote:
>> > In short, bug 6202130 is about removal of comments in Attributes
>> > like
>> >
>> > > XXX Need to handle UTF8 values and break up lines longer than 72
>> >
>> > The patch here contains a test that shows that utf characters are
>> > handled correctly, which has been the case before. Breaking lines
>> > has
>> > also been working before.
>> >
>> > In the main code, the patch only removes a few comments and does
>> > not
>> > change actual code. It all has been working before and now we know.
>> >
>> > Philipp
>> >
>> >
>> > https://bugs.openjdk.java.net/browse/JDK-6202130
>> >
>> >
>> >
>> >
>> > On Wed, 2018-09-26 at 07:08 +0200, Philipp Kunz wrote:
>> > > Hi,
>> > >
>> > > There are some comments in the source code like this:
>> > >
>> > > > XXX Need to handle UTF8 values and break up lines longer
>> > > than 72 bytes
>> > >
>> > > in Attributes.java on lines 299, 327, and 370. Bug 6202130 also
>> > > suggests
>> > > to add a test to show that the character set is fully supported
>> > > before
>> > > actually removing those comments.
>> > >
>> > > Hence, I added such a test, see ValueUtfEncoding in attached
>> > > patch, and
>> > > it finds the complete utf character set supported correctly and
>> > > that it
>> > > has been so already before.
>> > >
>> > > The comments in Attributes also say that lines longer than 72
>> > > bytes need
>> > > to be broken. That is a different issue and has been already
>> > > addressed
>> > > and solved in bug 6372077 and is now tested in LineBreakLineWidth
>> > > and
>> > > therefore, that line break part of the comment is obsolete now as
>> > > well
>> > > with the exception of version headers ("Manifest-Version" or
>> > > "Signature-Version") at least when taking strictly.
>> > >
>> > > Version headers are not subject of bug 6202130 for which my patch
>> > > intends to provide a fix and if only because multi-byte
>> > > characters are
>> > > not allowed in version headers and wasn't subject of 6372077
>> > > either
>> > > which was about too short lines rather than too long ones. If I
>> > > would
>> > > change only the minimum I could argue was safe, there would have
>> > > to
>> > > remain a comment in Attributes.java on line 327 like
>> > >
>> > > > XXX Need to handle version headers longer than 72 bytes
>> > >
>> > > but I guess that should probably better be represented as another
>> > > bug
>> > > rather than this style of comment if at all desired. There are
>> > > some bugs
>> > > concerning version headers and particularly the behavior of the
>> > > manifests in their absence, among possibly others, 6910466,
>> > > 4935610,
>> > > 4271239, 8196371 and 4256580 which have some relation but I
>> > > haven't
>> > > found one that really fits the 72 byte line length limit of
>> > > version
>> > > headers. Whether such a bug is created or found or not, it does
>> > > not
>> > > directly relate to the one I'm trying to fix here, 6202130.
>> > >
>> > > If also the test is good I would assume that the comments can be
>> > > removed now without actually changing the Manifest and Attributes
>> > > implementation.
>> > >
>> > > Regards,
>> > > Philipp
>>
Re: generated code and jigsaw modules
On 10/4/18 9:45 AM, Alan Bateman wrote: On 04/10/2018 17:10, Richard Hillegas wrote: I am looking for advice about how to tighten up module encapsulation while generating byte code on the fly. I ask this question on behalf of Apache Derby, a pure-Java relational database whose original code dates back to Java 1.2. I want to reduce Derby's attack-surface when running with a module path. First a little context: A relational database is an interpreter for the SQL language. It converts SQL queries into byte code which then runs on a virtual machine embedded in the interpreter. In Derby's case, the virtual machine is the Java VM and the byte code is simply Java byte code. That is, a Derby query plan is a class whose byte code is generated on the fly at run time. I have converted the Apache Derby codeline into a set of jigsaw modules: https://issues.apache.org/jira/browse/DERBY-6945. Unfortunately, I had to punch holes in the encapsulation of the main Derby module so that the generated query plans could call back into the Derby engine. That is because, by default, generated query plans load into the catch-all, unnamed module. Note that all of these generated classes live in a single package which does not belong to any named module. 1) Is it possible to load generated code into a named module? 2) Alternatively, can someone recommend another approach for preserving module encapsulation while generating classes on the fly? I would appreciate any advice or examples which you can recommend. There are a couple of places in the JDK where we spin bytecode into modules that are created at run-time. One example is in the Nashorn and was presented by MIchael Haupt at JVMLS 2017 [1]. There's a lot in that so a simpler example to look at is in the XML transformation code [2] where there is a module created at run-time for each translet. The module is fully encapsulate except for an entry point that it exports to the java.xml module in the parent module layer. In turn, the java.xml exports one of its internal packages to the translet module to allow what may be equivalent to your generated code calling back into the Derby engine. -Alan Thanks, Alan. I will study this example. Cheers! [1] https://www.youtube.com/watch?v=Zk6a6jNZAt0 [2] http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/src/java.xml/share/classes/com/sun/org/apache/xalan/internal/xsltc/trax/TemplatesImpl.java
Re: generated code and jigsaw modules
On 10/4/18 9:26 AM, Remi Forax wrote: - Mail original - De: "Richard Hillegas" À: "core-libs-dev" Envoyé: Jeudi 4 Octobre 2018 18:10:13 Objet: generated code and jigsaw modules I am looking for advice about how to tighten up module encapsulation while generating byte code on the fly. I ask this question on behalf of Apache Derby, a pure-Java relational database whose original code dates back to Java 1.2. I want to reduce Derby's attack-surface when running with a module path. First a little context: A relational database is an interpreter for the SQL language. It converts SQL queries into byte code which then runs on a virtual machine embedded in the interpreter. In Derby's case, the virtual machine is the Java VM and the byte code is simply Java byte code. That is, a Derby query plan is a class whose byte code is generated on the fly at run time. I have converted the Apache Derby codeline into a set of jigsaw modules: https://issues.apache.org/jira/browse/DERBY-6945. Unfortunately, I had to punch holes in the encapsulation of the main Derby module so that the generated query plans could call back into the Derby engine. That is because, by default, generated query plans load into the catch-all, unnamed module. Note that all of these generated classes live in a single package which does not belong to any named module. 1) Is it possible to load generated code into a named module? 2) Alternatively, can someone recommend another approach for preserving module encapsulation while generating classes on the fly? I would appreciate any advice or examples which you can recommend. you can use Lookup.defineClass. Thanks, Rémi. That gives me something to google up. Cheers! Thanks, -Rick cheers, Rémi
Re: RFC: JEP JDK-8208089: Implement C++14 Language Features
> On Oct 4, 2018, at 4:35 AM, Volker Simonis wrote: > > Hi Tim, Jeff, > > Kim has assembled a quite detailed list of new C++ features intended for use > in HotSpot/OpenJDK (with links to the corresponding C++ standard) > > https://bugs.openjdk.java.net/browse/JDK-8208089 Note that this list is intended as a starting point. The JEP also describes a process for adding to the list of permitted features, and I anticipate that process will be used. There are several that I personally think would be good to add (perhaps with some limits or restrictions), but that I think may involve more detailed discussion. And other folks may have their favorite must have feature(s) that didn’t seem to me to be needed in this initial seed list.
Re: RFC: JEP JDK-8208089: Implement C++14 Language Features
> On Oct 4, 2018, at 9:17 AM, Aleksei Voitylov > wrote: > > Kim, > > Thank you for posting this. It's an important step to make. I wanted to > clarify a couple of points: > > 1. Since this is infrastructure JEP, is the version of JDK which will undergo > such changes going to be some future version or does it apply to past > versions as well? I'm concerned with how we can simplify security backports > to 8u which (I currently assume) is not subject to this change. Future version (perhaps as soon as JDK 12). I don't think there is any desire to backport this change. And yes, introducing the use of new language features will make backports even more interesting than they already are. That seems unavoidable if we're going to pursue this direction. > 2. Which versions of GCC do you tentatively consider at this point? Non-x86 > ports may have a problem upgrading to a specific version of GCC which the > shared code will use and may need additional lead time to adjust. I think our (ad hoc) testing has been limited to gcc 7.x. But looking at the gcc language conformance tables and release notes, gcc 5.x might be good enough, and gcc 6.x seems fairly likely sufficient. Of course, older versions are more likely to have bugs in some of these new features. Updating the minimum supported versions for gcc and clang are noted as TBD in the JEP.
Re: JEP JDK-8208089: Implement C++14 Language Features
> On Oct 4, 2018, at 2:59 AM, Lindenmaier, Goetz > wrote: > > Hi, > > I appreciate this is handled in a JEP and well communicated! > > XLc, the compiler we use for AIX, might be a bottleneck here. > > But we currently have no compiler-constraints by products on > our aix port of jdk12 (for jdk11 we must stay with the current > compiler xlc 12 which does not support C++11, and jdk11 even > should be compilable with aCC by HP for which we won't > get new versions!) > > We will update our compiler for jdk/jdk to the most recent Xlc > as soon as possible. > To my current knowledge, Xlc 14 was dropped, and xlc 16 > is to be released early 2019. It is supposed to support > C++ 11, and also some C++ 14 features. The information I've been able to find mostly discusses Linux support for XLC 16. Any information about "some C++14 features"? I haven't found any detail on that. I am working toward being able to target this for JDK 12, but realize that might not be reachable. Windows is already there, because of VS2017 support. There are a couple of minor patches needed for gcc and clang based builds; Linux-x64 and MacOSX-x64 have had a fair amount of ad hoc testing. We (Oracle) only switched over the Solaris/sparc builds to the necessary compiler version (Oracle Studio 12.6) very recently, and there are some issues still to be dealt with there. And the currently used XLC is just not up to the change.
Re: [12] RFR: 8211398: Square character support for the Japanese new era
Hi Naoto, Looks ok to me. Regards, Roger On 10/04/2018 04:30 PM, Naoto Sato wrote: Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8211398 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8211398/webrev.00/ This change is to allow the code point (U+32FF) which will designate the square character for the Japanese new era. Naoto -- Thanks, Roger
[12] RFR: 8211398: Square character support for the Japanese new era
Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8211398 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8211398/webrev.00/ This change is to allow the code point (U+32FF) which will designate the square character for the Japanese new era. Naoto
Re: Review Request: JDK-8206240: java.lang.Class.newInstance() is causing caller to leak
Hi Mandy, On 10/04/2018 07:45 PM, Alan Bateman wrote: On 03/10/2018 18:28, Mandy Chung wrote: : On 10/3/18 9:30 AM, Peter Levart wrote: Hi Mandy, I don't know if this matters though, but should Reflection.getCallerClass() ever return null, previous behavior was to throw NPE (from Reflection.verifyMemberAccess(...)), now the checks are skipped. This should only be observable if [Class, Constructor].newInstance() was called from JNI's newly attached thread I believe. Yes that's the case. Such case never works as it will get NPE. I didn't worry too much about it. Since you mentioned it, I should keep the existing behavior (throwing NPE): http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8206240/webrev.01 The JNI call with no java frames is a corner case but good not to change existing behavior. The updated webrev looks okay to me. -Alan Likewise. Regards, Peter
Re: Review Request: 6202130: java.util.jar.Attributes.writeMain() can't handle multi-byte chars
Thanks to Sherman's hint, I revised the test to use the terms unicode
character and utf encoding consistently and not utf character. Affects
only the test, mostly in comments and a few identifiers.
Philipp
On Wed, 2018-10-03 at 12:48 -0700, Xueming Shen wrote:
> +1
>
> only nit is it might be preferred to use "utf8" directly in the test
> instead of current "utf"
>
> -sherman
>
> On 10/03/2018 11:55 AM, Philipp Kunz wrote:
> > In short, bug 6202130 is about removal of comments in Attributes
> > like
> >
> > > XXX Need to handle UTF8 values and break up lines longer than 72
> >
> > The patch here contains a test that shows that utf characters are
> > handled correctly, which has been the case before. Breaking lines
> > has
> > also been working before.
> >
> > In the main code, the patch only removes a few comments and does
> > not
> > change actual code. It all has been working before and now we know.
> >
> > Philipp
> >
> >
> > https://bugs.openjdk.java.net/browse/JDK-6202130
> >
> >
> >
> >
> > On Wed, 2018-09-26 at 07:08 +0200, Philipp Kunz wrote:
> > > Hi,
> > >
> > > There are some comments in the source code like this:
> > >
> > > > XXX Need to handle UTF8 values and break up lines longer
> > > than 72 bytes
> > >
> > > in Attributes.java on lines 299, 327, and 370. Bug 6202130 also
> > > suggests
> > > to add a test to show that the character set is fully supported
> > > before
> > > actually removing those comments.
> > >
> > > Hence, I added such a test, see ValueUtfEncoding in attached
> > > patch, and
> > > it finds the complete utf character set supported correctly and
> > > that it
> > > has been so already before.
> > >
> > > The comments in Attributes also say that lines longer than 72
> > > bytes need
> > > to be broken. That is a different issue and has been already
> > > addressed
> > > and solved in bug 6372077 and is now tested in LineBreakLineWidth
> > > and
> > > therefore, that line break part of the comment is obsolete now as
> > > well
> > > with the exception of version headers ("Manifest-Version" or
> > > "Signature-Version") at least when taking strictly.
> > >
> > > Version headers are not subject of bug 6202130 for which my patch
> > > intends to provide a fix and if only because multi-byte
> > > characters are
> > > not allowed in version headers and wasn't subject of 6372077
> > > either
> > > which was about too short lines rather than too long ones. If I
> > > would
> > > change only the minimum I could argue was safe, there would have
> > > to
> > > remain a comment in Attributes.java on line 327 like
> > >
> > > > XXX Need to handle version headers longer than 72 bytes
> > >
> > > but I guess that should probably better be represented as another
> > > bug
> > > rather than this style of comment if at all desired. There are
> > > some bugs
> > > concerning version headers and particularly the behavior of the
> > > manifests in their absence, among possibly others, 6910466,
> > > 4935610,
> > > 4271239, 8196371 and 4256580 which have some relation but I
> > > haven't
> > > found one that really fits the 72 byte line length limit of
> > > version
> > > headers. Whether such a bug is created or found or not, it does
> > > not
> > > directly relate to the one I'm trying to fix here, 6202130.
> > >
> > > If also the test is good I would assume that the comments can be
> > > removed now without actually changing the Manifest and Attributes
> > > implementation.
> > >
> > > Regards,
> > > Philipp
> diff -r d63efc278e93 src/java.base/share/classes/java/util/jar/Attributes.java
--- a/src/java.base/share/classes/java/util/jar/Attributes.java Thu Oct 04 10:19:01 2018 -0700
+++ b/src/java.base/share/classes/java/util/jar/Attributes.java Thu Oct 04 21:43:09 2018 +0200
@@ -296,7 +296,6 @@
/*
* Writes the current attributes to the specified data output stream.
- * XXX Need to handle UTF8 values and break up lines longer than 72 bytes
*/
@SuppressWarnings("deprecation")
void write(DataOutputStream os) throws IOException {
@@ -323,8 +322,6 @@
* Writes the current attributes to the specified data output stream,
* make sure to write out the MANIFEST_VERSION or SIGNATURE_VERSION
* attributes first.
- *
- * XXX Need to handle UTF8 values and break up lines longer than 72 bytes
*/
@SuppressWarnings("deprecation")
void writeMain(DataOutputStream out) throws IOException
@@ -367,7 +364,6 @@
/*
* Reads attributes from the specified input stream.
- * XXX Need to handle UTF8 values.
*/
void read(Manifest.FastInputStream is, byte[] lbuf) throws IOException {
read(is, lbuf, null, 0);
diff -r d63efc278e93 test/jdk/java/util/jar/Manifest/ValueUtfEncoding.java
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/test/jdk/java/util/jar/Manifest/ValueUtfEncoding.java Thu Oct 04 21:43:09 2018 +0200
@@ -0,0 +1,194 @@
+/*
+ * Copyright (c) 2018, Oracle and/or its
Re: Review Request: JDK-8206240: java.lang.Class.newInstance() is causing caller to leak
On 03/10/2018 18:28, Mandy Chung wrote: : On 10/3/18 9:30 AM, Peter Levart wrote: Hi Mandy, I don't know if this matters though, but should Reflection.getCallerClass() ever return null, previous behavior was to throw NPE (from Reflection.verifyMemberAccess(...)), now the checks are skipped. This should only be observable if [Class, Constructor].newInstance() was called from JNI's newly attached thread I believe. Yes that's the case. Such case never works as it will get NPE. I didn't worry too much about it. Since you mentioned it, I should keep the existing behavior (throwing NPE): http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8206240/webrev.01 The JNI call with no java frames is a corner case but good not to change existing behavior. The updated webrev looks okay to me. -Alan
Re: generated code and jigsaw modules
On 04/10/2018 17:10, Richard Hillegas wrote: I am looking for advice about how to tighten up module encapsulation while generating byte code on the fly. I ask this question on behalf of Apache Derby, a pure-Java relational database whose original code dates back to Java 1.2. I want to reduce Derby's attack-surface when running with a module path. First a little context: A relational database is an interpreter for the SQL language. It converts SQL queries into byte code which then runs on a virtual machine embedded in the interpreter. In Derby's case, the virtual machine is the Java VM and the byte code is simply Java byte code. That is, a Derby query plan is a class whose byte code is generated on the fly at run time. I have converted the Apache Derby codeline into a set of jigsaw modules: https://issues.apache.org/jira/browse/DERBY-6945. Unfortunately, I had to punch holes in the encapsulation of the main Derby module so that the generated query plans could call back into the Derby engine. That is because, by default, generated query plans load into the catch-all, unnamed module. Note that all of these generated classes live in a single package which does not belong to any named module. 1) Is it possible to load generated code into a named module? 2) Alternatively, can someone recommend another approach for preserving module encapsulation while generating classes on the fly? I would appreciate any advice or examples which you can recommend. There are a couple of places in the JDK where we spin bytecode into modules that are created at run-time. One example is in the Nashorn and was presented by MIchael Haupt at JVMLS 2017 [1]. There's a lot in that so a simpler example to look at is in the XML transformation code [2] where there is a module created at run-time for each translet. The module is fully encapsulate except for an entry point that it exports to the java.xml module in the parent module layer. In turn, the java.xml exports one of its internal packages to the translet module to allow what may be equivalent to your generated code calling back into the Derby engine. -Alan [1] https://www.youtube.com/watch?v=Zk6a6jNZAt0 [2] http://hg.openjdk.java.net/jdk/jdk/raw-file/tip/src/java.xml/share/classes/com/sun/org/apache/xalan/internal/xsltc/trax/TemplatesImpl.java
Re: generated code and jigsaw modules
- Mail original - > De: "Richard Hillegas" > À: "core-libs-dev" > Envoyé: Jeudi 4 Octobre 2018 18:10:13 > Objet: generated code and jigsaw modules > I am looking for advice about how to tighten up module encapsulation > while generating byte code on the fly. I ask this question on behalf of > Apache Derby, a pure-Java relational database whose original code dates > back to Java 1.2. I want to reduce Derby's attack-surface when running > with a module path. > > First a little context: A relational database is an interpreter for the > SQL language. It converts SQL queries into byte code which then runs on > a virtual machine embedded in the interpreter. In Derby's case, the > virtual machine is the Java VM and the byte code is simply Java byte > code. That is, a Derby query plan is a class whose byte code is > generated on the fly at run time. > > I have converted the Apache Derby codeline into a set of jigsaw modules: > https://issues.apache.org/jira/browse/DERBY-6945. Unfortunately, I had > to punch holes in the encapsulation of the main Derby module so that the > generated query plans could call back into the Derby engine. That is > because, by default, generated query plans load into the catch-all, > unnamed module. Note that all of these generated classes live in a > single package which does not belong to any named module. > > 1) Is it possible to load generated code into a named module? > > 2) Alternatively, can someone recommend another approach for preserving > module encapsulation while generating classes on the fly? > > I would appreciate any advice or examples which you can recommend. you can use Lookup.defineClass. > > Thanks, > -Rick cheers, Rémi
Re: [PATCH] 4511638: Double.toString(double) sometimes produces incorrect results
On 2018-10-04 15:28, Ulf Adams wrote:
> On Thu, Sep 27, 2018 at 5:37 PM Raffaello Giulietti
> mailto:raffaello.giulie...@gmail.com>>
> wrote:
>
> Hi Ulf,
>
>
> On 2018-09-27 16:40, Ulf Adams wrote:
> > Hi Raffaello,
> >
> > I am the author of a recent publication on double to string
> conversion
> > [1] - the Ryu algorithm. I've been aware of the problems with the Jdk
> > for several years, and am very much looking forward to
> improvements in
> > correctness and performance in this area.
> >
>
> What a coincidence! I'm happy to hear that the quest for better
> floating->string conversions has not stopped. Tomorrow I'll download
> your paper and have a look at it during the weekend.
>
>
> Have you had a chance to take a look?
>
> (I'm traveling for the next ~10 days and at a conference, so don't
> expect too much from me during that time.)
>
>
I had a cursory reading but couldn't dig deeper for now. If nothing
unexpected happens, I should be able to study your paper during the weekend.
>
>
>
> > I have done some testing against my Java implementation of the Ryu
> > algorithm described in the linked paper. Interestingly, I've found
> a few
> > cases where they output different results. In particular:
> > 1.0E-323 is printed as 9.9E-324
> > 1.0E-322 is printed as 9.9E-323
>
> If Ryu also produces 1 digit long outputs, then your results above are
> correct. But then Ryu should also output 5.0E-324 rather than 4.9E-324,
> for example.
> Even better, it should output 5E-324, 1E-323 and 1E-322 because adding
> the .0 part might confuse a human reader to believe that 2 digits are
> really needed. But then 4.9E-324, 9.9E-324 and 9.9E-323 are closer to
> the double.
>
>
> The C version produces 1 digit long outputs, and I was trying to follow
> the Java spec in the Java version, but the code to do so isn't quite
> right. Unfortunately, I haven't yet been able to fix it.
>
>
>
> 2 digits are for backward compatibility with the existing spec which
> requires at least one digit to the right of the decimal point.
>
>
> >
> > It's likely that there are more such cases - I only ran a sample of
> > double-precision numbers. Arguably, 9.9 is the correctly rounded
> 2-digit
> > output and Ryu is incorrect here. That's what you get when you have a
> > special case for Java without a correctness proof. :-(
> >
> > In terms of performance, this algorithm performs almost exactly
> the same
> > as my Java implementation of Ryu, although I'd like to point out
> that my
> > C implementation of Ryu is quite a bit faster (though note that it
> > generates different output, in particular, it only outputs a single
> > digit of precision in the above cases, rather than two), and I didn't
> > backport all the performance improvements from the Java version,
> yet. It
> > looks like this is not coincidence - as far as I can see so far, it's
> > algorithmically very similar, although it manages to avoid the
> loop I'm
> > using in Ryu to find the shortest representation.
> >
> > I have a few comments:
> >
> > * It rounds to {@code v} according to the usual
> > round-to-closest
> > * rule of IEEE 754 floating-point arithmetic.
> > - Since you're spelling out the rounding rules just below, this is
> > duplicated, and by itself, it's unclear since it doesn't specify the
> > specific sub-type (round half even).
> >
>
> I tried to save as much of the original spec wording as possible.
> Perhaps it isn't worthwhile.
>
>
>
> > - Naming: I'd strongly suggest to use variable names that relate to
> > what's stored, e.g., m for mantissa, e for exponent, etc.
> >
>
> I currently prefer to be consistent with a forthcoming paper of mine on
> the subject. But thanks for the suggestion.
>
>
> May I suggest that the paper also uses names that relate to what they're
> referring to? :-) Not that I've managed to do that very well myself...
>
I tend to use short "mathematical" names that still evoke their
semantics. Will see if I manage to be consistent.
>
>
>
> > - What's not clear to me is how the algorithm determines how many
> digits
> > to print.
> >
>
> You'll have to wait for the paper.
>
>
> Looking forward to it. I tried to reverse engineer the code, but it's
> far from obvious.
>
I don't think it is reversible, even for knowledgeable people like you :-(
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
I have to write the paper...
>
>
>
> > - Also, it might
generated code and jigsaw modules
I am looking for advice about how to tighten up module encapsulation while generating byte code on the fly. I ask this question on behalf of Apache Derby, a pure-Java relational database whose original code dates back to Java 1.2. I want to reduce Derby's attack-surface when running with a module path. First a little context: A relational database is an interpreter for the SQL language. It converts SQL queries into byte code which then runs on a virtual machine embedded in the interpreter. In Derby's case, the virtual machine is the Java VM and the byte code is simply Java byte code. That is, a Derby query plan is a class whose byte code is generated on the fly at run time. I have converted the Apache Derby codeline into a set of jigsaw modules: https://issues.apache.org/jira/browse/DERBY-6945. Unfortunately, I had to punch holes in the encapsulation of the main Derby module so that the generated query plans could call back into the Derby engine. That is because, by default, generated query plans load into the catch-all, unnamed module. Note that all of these generated classes live in a single package which does not belong to any named module. 1) Is it possible to load generated code into a named module? 2) Alternatively, can someone recommend another approach for preserving module encapsulation while generating classes on the fly? I would appreciate any advice or examples which you can recommend. Thanks, -Rick
Re: [PATCH] 4511638: Double.toString(double) sometimes produces incorrect results
On 2018-10-03 11:12, Andrew Dinn wrote:
> On 02/10/18 19:17, Martin Buchholz wrote:
>> Raffaello and Ulf should talk.
>> It seems like Ulf's ryu project is trying to solve the same problem.
>> ryu is also still being worked on, but there is already a published paper
>> and ryu is being adopted by various core libraries.
>> https://github.com/ulfjack/ryu
>> https://dl.acm.org/citation.cfm?id=3192369
>> Ulf, if you haven't already signed an Oracle Contributor Agreement for
>> openjdk, you should do so.
>> (Who knew printing floating point numbers could be so hard?)
>
> Well, Guy Steele and Jon L White, for starters (quite literally) [1]
>
In my eyes, the clarity of that paper is almost mythical.
I wish I could follow their teaching myself.
> [1] https://dl.acm.org/citation.cfm?id=93559
>
> regards,
>
>
> Andrew Dinn
> ---
> Senior Principal Software Engineer
> Red Hat UK Ltd
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
>
Re: [PATCH] 4511638: Double.toString(double) sometimes produces incorrect results
Hi, On 2018-10-02 20:17, Martin Buchholz wrote: > Raffaello and Ulf should talk. I work on this project in my spare time, which is limited and not entirely devoted to programming for the OpenJDK, of course ;-) Starting this weekend, I'll try to recollect my notes and organize them in paper or tutorial form. I cannot speak for Ulf, of course, but my understanding is that his porting to Java is a spare-time project, too. > It seems like Ulf's ryu project is trying to solve the same problem. > ryu is also still being worked on, but there is already a published > paper and ryu is being adopted by various core libraries. > https://github.com/ulfjack/ryu > https://dl.acm.org/citation.cfm?id=3192369 > Ulf, if you haven't already signed an Oracle Contributor Agreement for > openjdk, you should do so. > (Who knew printing floating point numbers could be so hard?) > Well, printing floating point numbers is simple if there are no performance concerns. Abstractly, it is only a matter of calculations with rational numbers. It becomes harder when efficiency becomes part of the goal. The quest started about 30 years ago with a seminal paper of Steele & White and seemed settled with an implementation by Gay around 1991. Since then, other interesting approaches have seen light with the goal of even better performance. Greetings Raffaello
RFR 8207690: SearchPath API for classpath and similar path strings
Please review a revised API for parsing search and using paths. Thanks for the earlier comments and suggestions. java.util.SearchPath is an immutable sequence of strings, typically filesystem paths. A search path is created from a list of strings or by splitting a string using File.pathSeparator. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-searchpath-8207690/ Javadoc: http://cr.openjdk.java.net/~rriggs/searchpath-8208208/java.base/java/util/SearchPath.html Issue: https://bugs.openjdk.java.net/browse/JDK-8207690 CSR: https://bugs.openjdk.java.net/browse/JDK-8208208 Thanks, Roger
Re: Fwd: Re: RFR (12): 8191053: Provide a mechanism to make system's security manager immutable
On Wed, Oct 3, 2018 at 7:53 PM Sergey Bylokhov wrote: > Hi, Sean. > One question related to SecurityManager and performance, is it possible > to provide a special version of AccessController.doPrivileged which will > be noop if SecurityManager is not present? TBH that method (at least, the no-arg variant) should *always* have been a no-op. All it really accomplishes, in practice, is to place a mark on the stack where the stack crawl to build the access control context should stop (plus one frame), and this effect is something that is already a natural consequence of a method being called in the JVM. The doPrivileged variant that accepts an ACC parameter should likewise always have been no-op other than stashing the nested ACC into some sort of thread-local (or better, a field on Thread) which can be referred to by the aforementioned stack crawl. The pure-java AccessController I prototyped late last year relies on these ideas, among other things. -- - DML
Re: [PATCH] 4511638: Double.toString(double) sometimes produces incorrect results
On Thu, Sep 27, 2018 at 5:37 PM Raffaello Giulietti <
raffaello.giulie...@gmail.com> wrote:
> Hi Ulf,
>
>
> On 2018-09-27 16:40, Ulf Adams wrote:
> > Hi Raffaello,
> >
> > I am the author of a recent publication on double to string conversion
> > [1] - the Ryu algorithm. I've been aware of the problems with the Jdk
> > for several years, and am very much looking forward to improvements in
> > correctness and performance in this area.
> >
>
> What a coincidence! I'm happy to hear that the quest for better
> floating->string conversions has not stopped. Tomorrow I'll download
> your paper and have a look at it during the weekend.
>
Have you had a chance to take a look?
(I'm traveling for the next ~10 days and at a conference, so don't expect
too much from me during that time.)
>
>
>
> > I have done some testing against my Java implementation of the Ryu
> > algorithm described in the linked paper. Interestingly, I've found a few
> > cases where they output different results. In particular:
> > 1.0E-323 is printed as 9.9E-324
> > 1.0E-322 is printed as 9.9E-323
>
> If Ryu also produces 1 digit long outputs, then your results above are
> correct. But then Ryu should also output 5.0E-324 rather than 4.9E-324,
> for example.
> Even better, it should output 5E-324, 1E-323 and 1E-322 because adding
> the .0 part might confuse a human reader to believe that 2 digits are
> really needed. But then 4.9E-324, 9.9E-324 and 9.9E-323 are closer to
> the double.
>
The C version produces 1 digit long outputs, and I was trying to follow the
Java spec in the Java version, but the code to do so isn't quite right.
Unfortunately, I haven't yet been able to fix it.
>
> 2 digits are for backward compatibility with the existing spec which
> requires at least one digit to the right of the decimal point.
>
>
> >
> > It's likely that there are more such cases - I only ran a sample of
> > double-precision numbers. Arguably, 9.9 is the correctly rounded 2-digit
> > output and Ryu is incorrect here. That's what you get when you have a
> > special case for Java without a correctness proof. :-(
> >
> > In terms of performance, this algorithm performs almost exactly the same
> > as my Java implementation of Ryu, although I'd like to point out that my
> > C implementation of Ryu is quite a bit faster (though note that it
> > generates different output, in particular, it only outputs a single
> > digit of precision in the above cases, rather than two), and I didn't
> > backport all the performance improvements from the Java version, yet. It
> > looks like this is not coincidence - as far as I can see so far, it's
> > algorithmically very similar, although it manages to avoid the loop I'm
> > using in Ryu to find the shortest representation.
> >
> > I have a few comments:
> >
> > * It rounds to {@code v} according to the usual
> > round-to-closest
> > * rule of IEEE 754 floating-point arithmetic.
> > - Since you're spelling out the rounding rules just below, this is
> > duplicated, and by itself, it's unclear since it doesn't specify the
> > specific sub-type (round half even).
> >
>
> I tried to save as much of the original spec wording as possible.
> Perhaps it isn't worthwhile.
>
>
>
> > - Naming: I'd strongly suggest to use variable names that relate to
> > what's stored, e.g., m for mantissa, e for exponent, etc.
> >
>
> I currently prefer to be consistent with a forthcoming paper of mine on
> the subject. But thanks for the suggestion.
>
May I suggest that the paper also uses names that relate to what they're
referring to? :-) Not that I've managed to do that very well myself...
>
>
> > - What's not clear to me is how the algorithm determines how many digits
> > to print.
> >
>
> You'll have to wait for the paper.
>
Looking forward to it. I tried to reverse engineer the code, but it's far
from obvious.
>
>
> > - Also, it might be nicer to move the long multiplications to a helper
> > method - at least from a short look, it looks like the computations of
> > vn, vnl, and vnr are identical.
> >
>
> I tried several variants: the current one seems to be the faster with
> the current optimizations of C2. Some day I'll also try with Graal.
>
Sure, but moving it to a method shouldn't affect performance (except if you
need to return multiple values), and, right now, it looks like identical
code.
>
>
>
> > - I looked through the spec, and it looks like all cases are
> > well-defined. Yay!
> >
> > I will need some more time to do a more thorough review of the code and
> > more testing for differences. Unfortunately, I'm also traveling the next
> > two weeks, so this might take a bit of time.
> >
>
> I thank you in advance for your willingness to review the code but my
> understanding is that only the officially appointed reviewers can
> approve OpenJDK contributions, which is of course a good policy.
> Besides, as two Andrews engineers from RedHat correctly observe,
> understanding the rationale of the code
Re: Fwd: Re: RFR (12): 8191053: Provide a mechanism to make system's security manager immutable
On 10/3/18 8:52 PM, Sergey Bylokhov wrote:
Hi, Sean.
One question related to SecurityManager and performance, is it possible
to provide a special version of AccessController.doPrivileged which will
be noop if SecurityManager is not present?
Yes, it may be possible, and we have prototyped it. But I didn't want to
mix it up with this one as it has some different specification
challenges -- for example, the AccessController APIs can be used
independently of the SecurityManager. I plan to get back to this one
soon and hope to have something that can be reviewed a bit later.
Thanks,
Sean
On 03/10/2018 13:12, Sean Mullan wrote:
For those of you that are not also subscribed to security-dev, this is
mostly FYI, as the review is winding down, but if you have any
comments, let me know.
This change will add new token options ("allow" and "disallow") to the
java.security.manager system property. The "disallow" option is
intended to provide a potential performance boost for applications
that don't enable a SecurityManager, as there is a cost associated
with allowing a SecurityManager to enabled at runtime, even if it is
never enabled. The CSR provides a good summary of the issue and spec
changes: https://bugs.openjdk.java.net/browse/JDK-8203316
Thanks,
Sean
Forwarded Message
Subject: Re: RFR (12): 8191053: Provide a mechanism to make system's
security manager immutable
Date: Tue, 2 Oct 2018 11:34:09 -0400
From: Sean Mullan
Organization: Oracle Corporation
To: security Dev OpenJDK
Hello,
Thanks for all the comments so far, and the interesting discussions
about the future of the SecurityManager. We will definitely return to
those discussions in the near future, but for now I have a second
webrev ready for review for this enhancement:
http://cr.openjdk.java.net/~mullan/webrevs/8191053/webrev.01/
The main changes since the initial revision are:
1. System.setSecurityManager is no longer deprecated. This type of
change clearly needs more discussion and is not an essential part of
this RFE.
2. After further thought, I took Bernd's suggestion [1] and instead of
adding a new property to disallow the setting of a SecurityManager at
runtime, have added new tokens to the existing "java.security.manager"
system property, named "=disallow", and "=allow" to toggle this
behavior. The "=" is to avoid any potential clashes with custom SM
classes with those names. I think this is a cleaner approach. There
are a couple of new paragraphs in the SecurityManager class
description describing the "java.security.manager" property and how
the new tokens work.
3. I also added a comment that Bernd had requested [2] on why
System.setSecurityManager calls checkPackageAccess("java.lang").
Also, the CSR has been updated:
https://bugs.openjdk.java.net/browse/JDK-8203316
Thanks,
Sean
[1]
http://mail.openjdk.java.net/pipermail/security-dev/2018-September/018217.html
[2]
http://mail.openjdk.java.net/pipermail/security-dev/2018-September/018193.html
On 9/13/18 4:02 PM, Sean Mullan wrote:
This is a SecurityManager related change which warrants some
additional details for its motivation.
The current System.setSecurityManager() API allows a SecurityManager
to be set at run-time. However, because of this mutability, it incurs
a performance overhead even for applications that never call it and
do not enable a SecurityManager dynamically, which is probably the
majority of applications.
For example, there are lots of "SecurityManager sm =
System.getSecurityManager(); if (sm != null) ..." checks in the JDK.
If it was known that a SecurityManager could never be set at
run-time, these checks could be optimized using constant-folding.
There are essentially two main parts to this change:
1. Deprecation of System.securityManager()
Going forward, we want to discourage applications from calling
System.setSecurityManager(). Instead they should enable a
SecurityManager using the java.security.manager system property on
the command-line.
2. A new JDK-specific system property to disallow the setting of the
security manager at run-time: jdk.allowSecurityManager
If set to false, it allows the run-time to optimize the code and
improve performance when it is known that an application will never
run with a SecurityManager. To support this behavior, the
System.setSecurityManager() API has been updated such that it can
throw an UnsupportedOperationException if it does not allow a
security manager to be set dynamically.
webrev: http://cr.openjdk.java.net/~mullan/webrevs/8191053/webrev.00/
CSR: https://bugs.openjdk.java.net/browse/JDK-8203316
JBS: https://bugs.openjdk.java.net/browse/JDK-8191053
(I will likely also send this to core-libs for additional review later)
--Sean
Re: RFC: JEP JDK-8208089: Implement C++14 Language Features
> 4 okt. 2018 kl. 10:57 skrev Martijn Verburg : > > Hi Kim, > > I like this initiative. I'm wondering if some of these rules can be easily > codified or written into a jcheck style checker (ccheck?) so that Authors > can adhere to the conventions and not rely on a Human review to pick out > where that convention isn't met. That's an interesting thought! I googled around a bit, but could find no obvious candidate for doing such a check. In fact, parsing and analyzing C++ seems quite a hard problem, compared to many other languages. (Sad, but not really surprising.) I found two possible routes to explore: cpplint [1], the official Google tool to verify that the Google C++ Style Guide [2] is followed, and Vera++ [3], a framework for creating scripts that can analyze and/or transform C++ code. Neither seem like an ideal solution. The Google tool is hard coded to support the Google rules. It's possible we can fork it and add our own, but it's not clear that this is even possible, much less so that it's a good way forward. Vera++, on the other hand, seems much more likely to be able to provide the tooling needed to write checks that enforce these rules. However, what we have is just a framework, and someone's got to write those rules (in TCL of all languages...). Then again, Vera++ is an old and quite popular tool, so its possible there is already a bunch of predefined rules that we can use as a starting point out there. (I didn't go that far in my analysis). Apart from these two, there appear to be no popular, well-encompassing, open source C++ checker out there, that could possibly be verifying rules like these. :( /Magnus [1] https://github.com/google/styleguide/tree/gh-pages/cpplint [2] https://google.github.io/styleguide/cppguide.html [3] https://bitbucket.org/verateam/vera/wiki/Introduction > > Cheers, > Martijn > > >> On Wed, 3 Oct 2018 at 20:13, Kim Barrett wrote: >> >> I've submitted a JEP for >> >> (1) enabling the use of C++14 Language Features when building the JDK, >> >> (2) define a process for deciding and documenting which new features >> can be used or are forbidden in HotSpot code, >> >> (3) provide an initial list of permitted and forbidden new features. >> >> https://bugs.openjdk.java.net/browse/JDK-8208089 >> >>
JarFile constructor throws undocumented exception (only) on Windows OS
Please consider the following trivial code:
import java.util.jar.*;
import java.io.*;
public class JarFileTest {
public static void main(final String[] args) throws Exception {
final String tmpDir = System.getProperty("java.io.tmpdir");
try {
final JarFile jarFile = new JarFile(tmpDir + File.separator
+ "*");
} catch (IOException ioe) {
System.out.println("Got the expected IOException " + ioe);
}
}
}
The constructor of the JarFile is passed a string which intentionally is
an (wildcard) invalid path. The above code (as expected) throws an
IOException on *nix systems across various version of Java (tested
against Java 8, 11). The same code throws an IOException (as expected)
in Java 8 on Windows OS too. However, in Java 11, on Windows OS, this
code throws a different exception (java.nio.file.InvalidPathException)
which isn't IOException or a subclass of it:
Exception in thread "main" java.nio.file.InvalidPathException: Illegal
char <*> at index 38: C:\Users\someone\AppData\Local\Temp\1\*
at
java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
at
java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
at java.base/java.io.File.toPath(File.java:2290)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1220)
at
java.base/java.util.zip.ZipFile$CleanableResource.(ZipFile.java:727)
at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:845)
at java.base/java.util.zip.ZipFile.(ZipFile.java:245)
at java.base/java.util.zip.ZipFile.(ZipFile.java:175)
at java.base/java.util.jar.JarFile.(JarFile.java:341)
at java.base/java.util.jar.JarFile.(JarFile.java:312)
at java.base/java.util.jar.JarFile.(JarFile.java:251)
at JarFileTest.main(JarFileTest.java:8)
The javadoc of JarFile constructor(s) mentions that:
* @throws IOException if an I/O error has occurred
Given that the javadoc doesn't mention anything about this other
exception, would this throwing of java.nio.file.InvalidPathException be
considered a bug in the implementation?
If this indeed is considered a bug, it appears to be the code in
java.util.zip.ZipFile$Source.get method which calls File#toPath(), which
as per its javadoc is indeed expected to throw an
java.nio.file.InvalidPathException if a Path cannot be constructed out
of the File. Looking at the implementation of the underlying
java.nio.file.FileSystem on *nix and Windows, they differ when it comes
to whether or not the exception gets thrown (Unix one seems to always
return a result). But keeping the implementation of
java.nio.file.FileSystem aside, I guess the
java.util.zip.ZipFile$Source.get code needs to have a catch block for
the InvalidPathException to wrap that into a IOException? Something like:
# HG changeset patch
# User Jaikiran Pai
# Date 1538645309 -19800
# Thu Oct 04 14:58:29 2018 +0530
# Node ID ff1bfd8cc9a3b385716fa5191bb68989d552f87e
# Parent 8705c6d536c5c197d0210acccf145ebc48f90227
Wrap and throw an IOException when a InvalidPathException is thrown
diff --git a/src/java.base/share/classes/java/util/zip/ZipFile.java
b/src/java.base/share/classes/java/util/zip/ZipFile.java
--- a/src/java.base/share/classes/java/util/zip/ZipFile.java
+++ b/src/java.base/share/classes/java/util/zip/ZipFile.java
@@ -35,6 +35,7 @@
import java.lang.ref.Cleaner.Cleanable;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
+import java.nio.file.InvalidPathException;
import java.nio.file.attribute.BasicFileAttributes;
import java.nio.file.Files;
import java.util.ArrayDeque;
@@ -1218,8 +1219,13 @@
static Source get(File file, boolean toDelete) throws IOException {
- Key key = new Key(file,
- Files.readAttributes(file.toPath(),
BasicFileAttributes.class));
+ final Key key;
+ try {
+ key = new Key(file,
+ Files.readAttributes(file.toPath(),
BasicFileAttributes.class));
+ } catch (InvalidPathException ipe) {
+ throw new IOException(ipe);
+ }
Source src;
synchronized (files) {
src = files.get(key);
Any thoughts?
-Jaikiran
Re: RFC: JEP JDK-8208089: Implement C++14 Language Features
Hi Kim, I like this initiative. I'm wondering if some of these rules can be easily codified or written into a jcheck style checker (ccheck?) so that Authors can adhere to the conventions and not rely on a Human review to pick out where that convention isn't met. Cheers, Martijn On Wed, 3 Oct 2018 at 20:13, Kim Barrett wrote: > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 > >
Re: RFC: JEP JDK-8208089: Implement C++14 Language Features
Hi Tim, Jeff, Kim has assembled a quite detailed list of new C++ features intended for use in HotSpot/OpenJDK (with links to the corresponding C++ standard) https://bugs.openjdk.java.net/browse/JDK-8208089 Could you please provide a summary of which of these features are already available since which version of xlC for AIX and potentially escalate the implementation of the missing features within IBM’s xlC on AIX team. Thank you and best regards, Volker Kim Barrett schrieb am Mi. 3. Okt. 2018 um 22:13: > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 > >
RE: JEP JDK-8208089: Implement C++14 Language Features
Hi, I appreciate this is handled in a JEP and well communicated! XLc, the compiler we use for AIX, might be a bottleneck here. But we currently have no compiler-constraints by products on our aix port of jdk12 (for jdk11 we must stay with the current compiler xlc 12 which does not support C++11, and jdk11 even should be compilable with aCC by HP for which we won't get new versions!) We will update our compiler for jdk/jdk to the most recent Xlc as soon as possible. To my current knowledge, Xlc 14 was dropped, and xlc 16 is to be released early 2019. It is supposed to support C++ 11, and also some C++ 14 features. Best regards, Goetz. > -Original Message- > From: core-libs-dev On Behalf > Of Kim Barrett > Sent: Mittwoch, 3. Oktober 2018 21:13 > To: hotspot-dev developers > Cc: build-dev ; core-libs- > d...@openjdk.java.net > Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features > > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089
