Re: JDK9 approach to limiting package to one module
On 30/03/2017 03:41, Stephen Felts wrote: We ran into a problem today caused by the JDK 9 single-module package limitation. We have an Apache jar file that references org.w3c.dom.ls.DocumentLS that is in another jar on the classpath. However, the incomplete org.w3c.dom.ls package is also in the JDK so the JDK wins and we get a class definition not found. This works on JDK 8 but not JDK 9. The only way to get around this without using a patch jar (I assume that might work) is to repackage org.w3c.dom in our jar files. Is there another solution? Can we get a complete package added to the JDK? org.w3c.dom was one of the Endorsed Standards APIs in Java SE 8 and older and so was upgradeable via the endorsed standards override mechanism. It was never upgradeable via the class path. As regards DocumentLS then I think this needs someone familiar with this API to comment. I checked the Java Language Bindings for the DOM L3 Load and Save spec but I don't see it. I do see drafts from 2002/2003 that reference it, is it possible that this interface didn't make the final release? -Alan
RE: JDK9 approach to limiting package to one module
Apparently it's in some earlier versions of Xalan and Xerces. For example, there is JDK-4827955 NoClassDefFoundError: org/w3c/dom/ls/DocumentLS causing failures in xalan conf where it needed to be removed from jaxp. It was removed from Xalan going from 2.5.1 to 2.6.0 (http://depfind.sourceforge.net/Samples/xalan.html). Apparently we have an ancient Xerces jar file. So the options are: 1. Use a newer Xerces/Xalan. 2. Use a patch jar file to add it to the package. That seems like a gamble that everything else will line up with the JDK. 3. Repackage our version of these class files -Original Message- From: Alan Bateman Sent: Thursday, March 30, 2017 3:36 AM To: Stephen Felts; jigsaw-dev@openjdk.java.net Subject: Re: JDK9 approach to limiting package to one module On 30/03/2017 03:41, Stephen Felts wrote: > We ran into a problem today caused by the JDK 9 single-module package > limitation. > > We have an Apache jar file that references org.w3c.dom.ls.DocumentLS that is > in another jar on the classpath. However, the incomplete org.w3c.dom.ls > package is also in the JDK so the JDK wins and we get a class definition not > found. > This works on JDK 8 but not JDK 9. > The only way to get around this without using a patch jar (I assume that > might work) is to repackage org.w3c.dom in our jar files. > Is there another solution? Can we get a complete package added to the JDK? org.w3c.dom was one of the Endorsed Standards APIs in Java SE 8 and older and so was upgradeable via the endorsed standards override mechanism. It was never upgradeable via the class path. As regards DocumentLS then I think this needs someone familiar with this API to comment. I checked the Java Language Bindings for the DOM L3 Load and Save spec but I don't see it. I do see drafts from 2002/2003 that reference it, is it possible that this interface didn't make the final release? -Alan
hg: jigsaw/jake/hotspot: 8177827: Hotspot changes to move java.lang.reflect.{Module, Layer, ...} to java.lang
Changeset: 6f3a68d98adb Author:hseigel Date: 2017-03-30 09:16 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/6f3a68d98adb 8177827: Hotspot changes to move java.lang.reflect.{Module,Layer,...} to java.lang Summary: Make the needed changes to hotspot source and all hotspot tests. Reviewed-by: alanb ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.core.common/src/org/graalvm/compiler/core/common/util/ModuleAPI.java ! src/share/vm/classfile/classFileParser.cpp ! src/share/vm/classfile/javaClasses.cpp ! src/share/vm/classfile/javaClasses.hpp ! src/share/vm/classfile/javaClasses.inline.hpp ! src/share/vm/classfile/moduleEntry.cpp ! src/share/vm/classfile/moduleEntry.hpp ! src/share/vm/classfile/modules.cpp ! src/share/vm/classfile/modules.hpp ! src/share/vm/classfile/systemDictionary.hpp ! src/share/vm/classfile/vmSymbols.hpp ! src/share/vm/oops/klass.cpp ! src/share/vm/prims/jvmtiEnv.cpp ! src/share/vm/runtime/reflection.cpp ! src/share/vm/runtime/thread.cpp ! test/compiler/jvmci/common/CTVMUtilities.java ! test/runtime/classFileParserBug/fakeMethodAccessor.jasm ! test/runtime/getSysPackage/GetSysPkgTest.java ! test/runtime/modules/JVMAddModuleExports.java ! test/runtime/modules/ModuleHelper.java ! test/runtime/modules/ModuleStress/src/jdk.test/test/Main.java ! test/runtime/modules/ModuleStress/src/jdk.test/test/MainGC.java ! test/runtime/modules/getModuleJNI/GetModule.java ! test/runtime/modules/java.base/java/lang/ModuleHelper.java ! test/serviceability/jdwp/AllModulesCommandTestDebuggee.java ! test/serviceability/jvmti/AddModuleExportsAndOpens/MyPackage/AddModuleExportsAndOpensTest.java ! test/serviceability/jvmti/AddModuleExportsAndOpens/libAddModuleExportsAndOpensTest.c ! test/serviceability/jvmti/AddModuleReads/MyPackage/AddModuleReadsTest.java ! test/serviceability/jvmti/AddModuleReads/libAddModuleReadsTest.c ! test/serviceability/jvmti/AddModuleUsesAndProvides/MyPackage/AddModuleUsesAndProvidesTest.java ! test/serviceability/jvmti/AddModuleUsesAndProvides/libAddModuleUsesAndProvidesTest.c ! test/serviceability/jvmti/GetModulesInfo/JvmtiGetAllModulesTest.java ! test/serviceability/jvmti/GetModulesInfo/libJvmtiGetAllModulesTest.c ! test/serviceability/jvmti/GetNamedModule/libGetNamedModuleTest.c
Disallowing the dynamic loading of agents by default
// Moving the general discussion to jigsaw-dev for the record; // bcc'ing {hotspot-runtime,serviceability}-dev for reference. Andrew, Thanks for your feedback on this topic [1][2][3]. First, we apologize for the way in which this topic was raised. Our intent was to post a proposal for discussion prior to code review, but unfortunately that review was posted prematurely (as is evident by its inclusion of Oracle-internal e-mail addresses and URLs). Second, I agree with your earlier analysis as to the security impact of this change. If an attack is possible via this vector then closing the vector would only slow the attack, not prevent it. The motivation for this change is, however, not merely to improve the security of the platform but to improve its integrity, which is one of the principal goals of the entire modularity effort. Integrity has benefits for security and also for maintainability, of both the JDK itself and of code that runs upon it, since it makes it clear exactly which APIs are intended for external use, and which are not. To improve platform integrity we've strongly encapsulated (most) of the internal APIs of the JDK. That can be, as we all know, a source of pain for developers trying to get existing applications to run on JDK 9, so we've provided workarounds via the encapsulation-busting `--add-opens` and `--add-exports` command-line options and, recently, the temporary but more powerful (and verbose) `--permit-illegal-access` option. There are no API equivalents to these command-line options [4]. That's intentional: We want developers and deployers using Java out-of-the-box to be assured that their code is only using APIs actually intended for external use. They can choose explicitly to expose internal APIs, via the command line, but then that's a deliberate choice that they make, and they own the consequences. It's reasonable to allow this via the command line since we assume that anyone with access to the command line already has the power to corrupt the JDK in any way they please. The proposal to disable the loading of dynamic agents by default is one more part of the overall integrity story. As things stand today, code in any JAR file can use the `VirtualMachine.loadAgent` API to load an agent into the JVM in which it's running and, via that agent, break into any module it likes. Changing the default and providing a new command-line option, in this case `-XX:+EnableDyanmicAgentLoading`, is consistent with the other encapsulation-busting options we've introduced. It allows a developer or deployer to choose to bend, if not violate, the integrity of the platform, yet also to know that by default the platform is whole. (Changing this default is also consistent, as Alan has noted, with the original intent of the `VirtualMachine.loadAgent` API.) I understand your points about the practical difficulties of having to educate users about this new option and enhance startup scripts to use the option only when invoking JDK 9. Isn't it already the case, however, that migrating existing applications to JDK 9 is often going to require the use of a few new options anyway, in order to expose internal APIs? If so then would it really be that much more burdensome for users also to think explicitly, at the same time, about whether they want to enable dynamic agent loading? This change would be disruptive to some but it's the best way we've found, so far, to preserve platform integrity in the face of dynamic agent loading. If there's a better way to do that, we'd like to know. - Mark [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-March/022948.html [2] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-March/022951.html [3] https://bugs.openjdk.java.net/browse/JDK-8177154 [4] There are limited reflective APIs via which code in a module can apply some of these operations to just that module, and via which code that creates a module layer can do so for modules in that layer, but none of these APIs can break into arbitrary modules.
hg: jigsaw/jake/jaxws: 2 new changesets
Changeset: 22b17dea56eb Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/22b17dea56eb Added tag jdk-9+163 for changeset 3890f96e8995 ! .hgtags Changeset: 1bbf42656eb0 Author:alanb Date: 2017-03-30 18:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxws/rev/1bbf42656eb0 Merge ! .hgtags
hg: jigsaw/jake/corba: 2 new changesets
Changeset: 48a6b541153d Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/48a6b541153d Added tag jdk-9+163 for changeset 493011dee80e ! .hgtags Changeset: e371a4b02dff Author:alanb Date: 2017-03-30 18:21 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/corba/rev/e371a4b02dff Merge ! .hgtags
hg: jigsaw/jake: 2 new changesets
Changeset: 41d9f0545d53 Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/rev/41d9f0545d53 Added tag jdk-9+163 for changeset c38c6b270ccc ! .hgtags Changeset: ae6006a61cd2 Author:alanb Date: 2017-03-30 18:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/rev/ae6006a61cd2 Merge ! .hgtags
hg: jigsaw/jake/langtools: 2 new changesets
Changeset: 4c09d6da5f6b Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/4c09d6da5f6b Added tag jdk-9+163 for changeset 24582dd2649a ! .hgtags Changeset: 31a0299184a9 Author:alanb Date: 2017-03-30 18:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/langtools/rev/31a0299184a9 Merge ! .hgtags
hg: jigsaw/jake/nashorn: 2 new changesets
Changeset: b473fab09baa Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/b473fab09baa Added tag jdk-9+163 for changeset 5e5e436543da ! .hgtags Changeset: d8794ca6e3dd Author:alanb Date: 2017-03-30 18:21 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/nashorn/rev/d8794ca6e3dd Merge ! .hgtags
hg: jigsaw/jake/hotspot: 5 new changesets
Changeset: 015084c7ef97 Author:mbaesken Date: 2017-03-25 00:00 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/015084c7ef97 8177531: libGetNamedModuleTest.c crash when printing NULL-pointer Summary: Fix the NULL-pointer issue Reviewed-by: stuefe, simonis, sspitsyn ! test/serviceability/jvmti/GetNamedModule/libGetNamedModuleTest.c Changeset: b0b56932255e Author:thartmann Date: 2017-03-27 10:12 +0200 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/b0b56932255e 8177095: Range check dependent CastII/ConvI2L is prematurely eliminated Summary: Disabled narrowing of range check dependent CastIIs (either through the CastII(AddI) optimization or through CastIINode::Ideal). Reviewed-by: vlivanov, kvn ! src/share/vm/opto/castnode.cpp ! src/share/vm/opto/convertnode.cpp ! test/compiler/loopopts/TestLoopPeeling.java Changeset: 983fe2075557 Author:adinn Date: 2017-03-27 06:18 -0400 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/983fe2075557 8177661: AArch64: Incorrect C2 patterns cause system register corruption Summary: Correct ad rule output register types from iRegX to iRegXNoSp Reviewed-by: aph, kvn ! src/cpu/aarch64/vm/aarch64.ad Changeset: 60721d6ff1ac Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/60721d6ff1ac Added tag jdk-9+163 for changeset 983fe2075557 ! .hgtags Changeset: 7e5cc8721fe5 Author:alanb Date: 2017-03-30 18:33 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/7e5cc8721fe5 Merge ! .hgtags ! test/serviceability/jvmti/GetNamedModule/libGetNamedModuleTest.c
hg: jigsaw/jake/jdk: 7 new changesets
Changeset: acd4fd0fd6e8 Author:bpb Date: 2017-03-24 14:46 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/acd4fd0fd6e8 8177564: Remove check for Windows XP and Server 2003 in java/nio/channels/DatagramChannel/NetworkConfiguration.java Summary: Remove check for XP and Server 2003 in IPv6 support determination. Reviewed-by: alanb ! test/java/nio/channels/DatagramChannel/NetworkConfiguration.java Changeset: 3bffd193a3a5 Author:bpb Date: 2017-03-24 15:02 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/3bffd193a3a5 8177565: java/nio/channels/Selector/SelectorLimit.java disabled for Windows release >= 6.0 Summary: Remove check of Windows version Reviewed-by: alanb ! test/java/nio/channels/Selector/SelectorLimit.java Changeset: fb54b256d751 Author:mchung Date: 2017-03-27 15:12 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/fb54b256d751 8174826: jlink support for linking in service provider modules Reviewed-by: alanb, anazarov ! src/jdk.jlink/share/classes/jdk/tools/jimage/JImageTask.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/Jlink.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JlinkTask.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/TaskHelper.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/packager/AppRuntimeImageBuilder.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/resources/jlink.properties ! test/tools/jlink/IntegrationTest.java ! test/tools/jlink/JLinkTest.java + test/tools/jlink/bindservices/BindServices.java + test/tools/jlink/bindservices/SuggestProviders.java + test/tools/jlink/bindservices/src/m1/module-info.java + test/tools/jlink/bindservices/src/m1/p1/Impl.java + test/tools/jlink/bindservices/src/m1/p1/Main.java + test/tools/jlink/bindservices/src/m1/p1/S.java + test/tools/jlink/bindservices/src/m2/module-info.java + test/tools/jlink/bindservices/src/m2/p2/Impl.java + test/tools/jlink/bindservices/src/m2/p2/T.java + test/tools/jlink/bindservices/src/m3/module-info.java + test/tools/jlink/bindservices/src/m3/p3/Impl.java Changeset: e685e3197f62 Author:darcy Date: 2017-03-27 18:38 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/e685e3197f62 8177678: Overstatement of universality of Era.getDisplayName() implementation Reviewed-by: naoto ! src/java.base/share/classes/java/time/chrono/Era.java ! src/java.base/share/classes/java/time/chrono/JapaneseEra.java Changeset: 50171f8c4796 Author:mli Date: 2017-03-27 18:52 -0700 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/50171f8c4796 8176865: overridden api has a wrong since value in java.base module Reviewed-by: alanb ! src/java.base/share/classes/java/lang/String.java ! src/java.base/share/classes/java/lang/reflect/Constructor.java ! src/java.base/share/classes/java/nio/MappedByteBuffer.java ! src/java.base/share/classes/java/nio/X-Buffer.java.template ! src/java.base/share/classes/java/security/SecureRandom.java ! src/java.base/share/classes/java/security/SecureRandomSpi.java Changeset: bd00098fc2d7 Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/bd00098fc2d7 Added tag jdk-9+163 for changeset 50171f8c4796 ! .hgtags Changeset: 415d9fa6fe82 Author:alanb Date: 2017-03-30 18:33 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/415d9fa6fe82 Merge ! .hgtags ! src/java.base/share/classes/java/lang/reflect/Constructor.java ! src/jdk.jlink/share/classes/jdk/tools/jimage/JImageTask.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/Jlink.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/JlinkTask.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/TaskHelper.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/internal/packager/AppRuntimeImageBuilder.java ! src/jdk.jlink/share/classes/jdk/tools/jlink/resources/jlink.properties ! test/tools/jlink/IntegrationTest.java ! test/tools/jlink/JLinkTest.java
hg: jigsaw/jake/jaxp: 2 new changesets
Changeset: fe0c5dfe9ce9 Author:lana Date: 2017-03-29 23:33 + URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/fe0c5dfe9ce9 Added tag jdk-9+163 for changeset 92a38c75cd27 ! .hgtags Changeset: 0a4b855cb7fd Author:alanb Date: 2017-03-30 18:20 +0100 URL: http://hg.openjdk.java.net/jigsaw/jake/jaxp/rev/0a4b855cb7fd Merge ! .hgtags
Re: Disallowing the dynamic loading of agents by default
Mark, what is the plan for people with .jar files on their desktop who just double click on them to run desktop applications? How will they know that their upgrade to Java 9 broke the application and how to remedy the problem by either getting a java-9 compatible version of the app (which will likely not be available at Java-9 release), or that they need to roll back to some previous version of Java until the application they are using is compatible with Java-9? It seems like there is an awful lot of focus on command line solutions when there is a whole class of users who do not use the command line. Gregg > On Mar 30, 2017, at 10:38 AM, mark.reinh...@oracle.com wrote: > > // Moving the general discussion to jigsaw-dev for the record; > // bcc'ing {hotspot-runtime,serviceability}-dev for reference. > > Andrew, > > Thanks for your feedback on this topic [1][2][3]. > > First, we apologize for the way in which this topic was raised. Our > intent was to post a proposal for discussion prior to code review, but > unfortunately that review was posted prematurely (as is evident by its > inclusion of Oracle-internal e-mail addresses and URLs). > > Second, I agree with your earlier analysis as to the security impact of > this change. If an attack is possible via this vector then closing the > vector would only slow the attack, not prevent it. > > The motivation for this change is, however, not merely to improve the > security of the platform but to improve its integrity, which is one of > the principal goals of the entire modularity effort. Integrity has > benefits for security and also for maintainability, of both the JDK > itself and of code that runs upon it, since it makes it clear exactly > which APIs are intended for external use, and which are not. > > To improve platform integrity we've strongly encapsulated (most) of the > internal APIs of the JDK. That can be, as we all know, a source of pain > for developers trying to get existing applications to run on JDK 9, so > we've provided workarounds via the encapsulation-busting `--add-opens` > and `--add-exports` command-line options and, recently, the temporary > but more powerful (and verbose) `--permit-illegal-access` option. > > There are no API equivalents to these command-line options [4]. That's > intentional: We want developers and deployers using Java out-of-the-box > to be assured that their code is only using APIs actually intended for > external use. They can choose explicitly to expose internal APIs, via > the command line, but then that's a deliberate choice that they make, and > they own the consequences. It's reasonable to allow this via the command > line since we assume that anyone with access to the command line already > has the power to corrupt the JDK in any way they please. > > The proposal to disable the loading of dynamic agents by default is one > more part of the overall integrity story. As things stand today, code in > any JAR file can use the `VirtualMachine.loadAgent` API to load an agent > into the JVM in which it's running and, via that agent, break into any > module it likes. Changing the default and providing a new command-line > option, in this case `-XX:+EnableDyanmicAgentLoading`, is consistent with > the other encapsulation-busting options we've introduced. It allows a > developer or deployer to choose to bend, if not violate, the integrity > of the platform, yet also to know that by default the platform is whole. > (Changing this default is also consistent, as Alan has noted, with the > original intent of the `VirtualMachine.loadAgent` API.) > > I understand your points about the practical difficulties of having to > educate users about this new option and enhance startup scripts to use > the option only when invoking JDK 9. Isn't it already the case, however, > that migrating existing applications to JDK 9 is often going to require > the use of a few new options anyway, in order to expose internal APIs? > If so then would it really be that much more burdensome for users also > to think explicitly, at the same time, about whether they want to enable > dynamic agent loading? > > This change would be disruptive to some but it's the best way we've > found, so far, to preserve platform integrity in the face of dynamic > agent loading. If there's a better way to do that, we'd like to know. > > - Mark > > > [1] > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-March/022948.html > [2] > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-March/022951.html > [3] https://bugs.openjdk.java.net/browse/JDK-8177154 > [4] There are limited reflective APIs via which code in a module can >apply some of these operations to just that module, and via which >code that creates a module layer can do so for modules in that >layer, but none of these APIs can break into arbitrary modules.
How to become a "cooperative troubleshooting tool"?
Hello all, I was quite surprised when I learned that on short notice a change landed to jdk9 that by default disallows agent runtime attach. The JDK-8177154 mentiones that the implementation is desigend to keep random agents out while still allowing "cooperative troubleshooting tools". Otherwise -XX:+DisableAttachMechanism would have been sufficient. As the vendor of troubleshooting and monitoring tools, we would like to know how we can be considered cooperative, to be still allowed to use the attach API. As of now, we use the attach API to monitor a vast number of JVMs in the wild. The description in the issue hints at that "self-elevation" of code via attach was the primary target, but that because such code could fork a new JVM to remote attach, this effectively cannot be prevented. I wonder why the self-elevation cannot be prevented using the SecurityManager? It would prevent VirtualMachine.loadAgent and variants of Runtime.exec. With disabling the attach API loadAgent() users no longer can use any monitoring or troubleshooting tool that uses javaagents that use instrumentation (that is not only us) Could vendors become certified to still be allowed to invoke the API? Is that the idea behind cooperative troubleshooting tools? I want to question the statement: 5. The change should only impact a small number of tools Has an evaluation been performed on what tools use the API and how many of them are affected? The notes also state that not resolving the jdk.attach module will not work because several tools use that API. I find this confusing. Aren't those tools also affected by turning off the loadAgent API? I would appreciate some information on this and how we can still function by default in the future. Best regards Fabian Lange -- Fabian Lange | VP of Engineering Mobile: +49 (0) 160.3673393 INSTANA Inc | 541 Jefferson Ave, Ste 100 | 94063 Redwood City, CA | United States INSTANA GMBH | Hochstraße 11 | 42697 Solingen | Deutschland Sitz der Gesellschaft: Solingen | HRB 26637 | Amtsgericht Wuppertal Geschäftsführer: Mirko Novakovic | Pavlo Baron
Re: How to become a "cooperative troubleshooting tool"?
There has been some discussion on this list which I have never been satisfied with around why the SecurityManager was not used for a large number of the things that are being done with literal JVM changes. Mark Reinhold and others, I believe, don’t know how to use the SecurityManager, or believe that it’s not sufficient for security, even though it has Security in its name. If the JVM is already compromised in security, even with a SecurityManager in place, it would be nice for Oracle to announce this and provide a solution. What seems interesting to me, is that the whole Jigsaw effort has been focused only on what the JDK needs for encapsulation and isolation, not on what the general Java community needs. They rolled this up, over time, with casual involvement of some community members and are on the home stretch, it seems, to subject all kinds of people to all kinds of pain and broken applications as soon as JDK-9 is released and people start mindlessly upgrading. This version of Java is breaking all kinds of software in all kinds of ways. My personal opinion is that this is really not an answer to much of anything. The security solutions are not real. You can still edit Jar files. You can still recast code with various tooling to provide access to exactly the APIs that they are trying to keep you from using. As long as there is JNI, there is a way into the JVM with native code to do anything. As long as there is no SecurityManager active, any jar can include as many native JNI code fragments as needed to access as much of the native computer environment as the developer desires. All of the things that they are attempting here are blind attempts at confining access to software services in the JDK while there is really no way to do that. Gregg > On Mar 30, 2017, at 2:49 PM, Fabian Lange wrote: > > Hello all, > I was quite surprised when I learned that on short notice a change > landed to jdk9 that by default disallows agent runtime attach. > The JDK-8177154 mentiones that the implementation is desigend to keep > random agents out while still allowing "cooperative troubleshooting > tools". Otherwise -XX:+DisableAttachMechanism would have been > sufficient. > > As the vendor of troubleshooting and monitoring tools, we would like > to know how we can be considered cooperative, to be still allowed to > use the attach API. > As of now, we use the attach API to monitor a vast number of JVMs in the wild. > > The description in the issue hints at that "self-elevation" of code > via attach was the primary target, but that because such code could > fork a new JVM to remote attach, this effectively cannot be prevented. > I wonder why the self-elevation cannot be prevented using the > SecurityManager? It would prevent VirtualMachine.loadAgent and > variants of Runtime.exec. > > With disabling the attach API loadAgent() users no longer can use any > monitoring or troubleshooting tool that uses javaagents that use > instrumentation (that is not only us) > Could vendors become certified to still be allowed to invoke the API? > Is that the idea behind cooperative troubleshooting tools? > > I want to question the statement: > 5. The change should only impact a small number of tools > > Has an evaluation been performed on what tools use the API and how > many of them are affected? > > The notes also state that not resolving the jdk.attach module will not > work because several tools use that API. I find this confusing. Aren't > those tools also affected by turning off the loadAgent API? > > I would appreciate some information on this and how we can still > function by default in the future. > > Best regards > Fabian Lange > > -- > Fabian Lange | VP of Engineering > Mobile: +49 (0) 160.3673393 > > INSTANA Inc | 541 Jefferson Ave, Ste 100 | 94063 Redwood City, CA | > United States > > INSTANA GMBH | Hochstraße 11 | 42697 Solingen | Deutschland > Sitz der Gesellschaft: Solingen | HRB 26637 | Amtsgericht Wuppertal > Geschäftsführer: Mirko Novakovic | Pavlo Baron
RE: Disallowing the dynamic loading of agents by default
I try to be supportive of the rather disruptive changes that Java 9 introduces but this one would really do more harm than good. I work a lot with customers that build tools, many of them would be affected by this change in a negative way. Did you evaluate the use of the dynamic attachment API? Dynamic attachment has been around for more than ten years and you are attempting to cripple this feature in a footnote. I really believe that you are underestimating the impacts of this change. The entire point of a Java agent is to attach to a process after it was already started. I know of numerous use cases where the need for an agent is only discovered after a Java process is already running and the attachment is used as a means to adjusting a process without restarting it. And please bear in mind that tools departments or vendors do often not control the deployment procedure for Java processes and cannot dicatate start-up parameters as the one you suggest. It is an important feature of a Java agent to work independently of the JVM's deployment which you are taking away. (This is in particularly true for microservice setups where there are too many JVMs and not interfering with the deployment supports dev-ops culture.) Security was always a big argument for justifing these change but as dynamic attachment requires running as the same user, there is no security benefit to gain in this context. And I do not see how the integrity argument adds a notable benefit. Migrating to Java 9 is already imposing a big challange and this change will make it even harder. Please do not add this constraint.
Re: Disallowing the dynamic loading of agents by default
Hi Mark, I understand the need for consistency and I agree with your rationale. I think though that a solution that requires a command line switch for such cases is too restrictive. The problem is that the main users of this API are generally agents that are injected after a particular problem may arise. Requiring a command line switch like this means that for those use cases you must restart the applications, or effectively always turn on the feature, which practically invalidates the need for this restriction. I wonder if the JDK should not instead have a default serviceability agent that can have a runtime impact on this flag, something more restricted than a generic JMX agent. I already see support engineers starting to recommend go to customers :) Seriously though, it seems to me that this restriction should be better thought and perhaps left for later (I understand that this means people will need to change their command lines twice,but this solution seems very rushed). Cheers, Mario On Thu 30. Mar 2017 at 17:39, wrote: > // Moving the general discussion to jigsaw-dev for the record; > // bcc'ing {hotspot-runtime,serviceability}-dev for reference. > > Andrew, > > Thanks for your feedback on this topic [1][2][3]. > > First, we apologize for the way in which this topic was raised. Our > intent was to post a proposal for discussion prior to code review, but > unfortunately that review was posted prematurely (as is evident by its > inclusion of Oracle-internal e-mail addresses and URLs). > > Second, I agree with your earlier analysis as to the security impact of > this change. If an attack is possible via this vector then closing the > vector would only slow the attack, not prevent it. > > The motivation for this change is, however, not merely to improve the > security of the platform but to improve its integrity, which is one of > the principal goals of the entire modularity effort. Integrity has > benefits for security and also for maintainability, of both the JDK > itself and of code that runs upon it, since it makes it clear exactly > which APIs are intended for external use, and which are not. > > To improve platform integrity we've strongly encapsulated (most) of the > internal APIs of the JDK. That can be, as we all know, a source of pain > for developers trying to get existing applications to run on JDK 9, so > we've provided workarounds via the encapsulation-busting `--add-opens` > and `--add-exports` command-line options and, recently, the temporary > but more powerful (and verbose) `--permit-illegal-access` option. > > There are no API equivalents to these command-line options [4]. That's > intentional: We want developers and deployers using Java out-of-the-box > to be assured that their code is only using APIs actually intended for > external use. They can choose explicitly to expose internal APIs, via > the command line, but then that's a deliberate choice that they make, and > they own the consequences. It's reasonable to allow this via the command > line since we assume that anyone with access to the command line already > has the power to corrupt the JDK in any way they please. > > The proposal to disable the loading of dynamic agents by default is one > more part of the overall integrity story. As things stand today, code in > any JAR file can use the `VirtualMachine.loadAgent` API to load an agent > into the JVM in which it's running and, via that agent, break into any > module it likes. Changing the default and providing a new command-line > option, in this case `-XX:+EnableDyanmicAgentLoading`, is consistent with > the other encapsulation-busting options we've introduced. It allows a > developer or deployer to choose to bend, if not violate, the integrity > of the platform, yet also to know that by default the platform is whole. > (Changing this default is also consistent, as Alan has noted, with the > original intent of the `VirtualMachine.loadAgent` API.) > > I understand your points about the practical difficulties of having to > educate users about this new option and enhance startup scripts to use > the option only when invoking JDK 9. Isn't it already the case, however, > that migrating existing applications to JDK 9 is often going to require > the use of a few new options anyway, in order to expose internal APIs? > If so then would it really be that much more burdensome for users also > to think explicitly, at the same time, about whether they want to enable > dynamic agent loading? > > This change would be disruptive to some but it's the best way we've > found, so far, to preserve platform integrity in the face of dynamic > agent loading. If there's a better way to do that, we'd like to know. > > - Mark > > > [1] > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-March/022948.html > [2] > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2017-March/022951.html > [3] https://bugs.openjdk.java.net/browse/JDK-8177154 > [4] There are l
Re: How to become a "cooperative troubleshooting tool"?
On 30/03/2017 20:49, Fabian Lange wrote: : The notes also state that not resolving the jdk.attach module will not work because several tools use that API. I find this confusing. Aren't those tools also affected by turning off the loadAgent API? The troubleshooting tools (jcmd, jstack, etc.) use the attach mechanism but aren't loading agents into the target VM so no impact. Ditto for management tools that start the JMX agent. -Alan
Re: Disallowing the dynamic loading of agents by default
Hi All, We're one of the APM tooling vendors (I suspect all of us are in the same boat) whose customers would be adversely impacted by this change. In effect a vital part of our tooling would not function without the proposed flag being set. The likely impact is that our customers would stay on Java 8 for longer until they were prepared to add the extra CL flag or lose a significant set of analysis features that our tooling provides. I don't have any proposed solutions right now, but we'll do some digging and see if we can help propose alternatives that might meet the same goal. Cheers, Martijn On 30 March 2017 at 21:55, Mario Torre wrote: > Hi Mark, > > I understand the need for consistency and I agree with your rationale. > > I think though that a solution that requires a command line switch for such > cases is too restrictive. > > The problem is that the main users of this API are generally agents that > are injected after a particular problem may arise. Requiring a command line > switch like this means that for those use cases you must restart the > applications, or effectively always turn on the feature, which practically > invalidates the need for this restriction. > > I wonder if the JDK should not instead have a default serviceability agent > that can have a runtime impact on this flag, something more restricted than > a generic JMX agent. > > I already see support engineers starting to recommend go to customers :) > > Seriously though, it seems to me that this restriction should be better > thought and perhaps left for later (I understand that this means people > will need to change their command lines twice,but this solution seems very > rushed). > > Cheers, > Mario > > On Thu 30. Mar 2017 at 17:39, wrote: > > > // Moving the general discussion to jigsaw-dev for the record; > > // bcc'ing {hotspot-runtime,serviceability}-dev for reference. > > > > Andrew, > > > > Thanks for your feedback on this topic [1][2][3]. > > > > First, we apologize for the way in which this topic was raised. Our > > intent was to post a proposal for discussion prior to code review, but > > unfortunately that review was posted prematurely (as is evident by its > > inclusion of Oracle-internal e-mail addresses and URLs). > > > > Second, I agree with your earlier analysis as to the security impact of > > this change. If an attack is possible via this vector then closing the > > vector would only slow the attack, not prevent it. > > > > The motivation for this change is, however, not merely to improve the > > security of the platform but to improve its integrity, which is one of > > the principal goals of the entire modularity effort. Integrity has > > benefits for security and also for maintainability, of both the JDK > > itself and of code that runs upon it, since it makes it clear exactly > > which APIs are intended for external use, and which are not. > > > > To improve platform integrity we've strongly encapsulated (most) of the > > internal APIs of the JDK. That can be, as we all know, a source of pain > > for developers trying to get existing applications to run on JDK 9, so > > we've provided workarounds via the encapsulation-busting `--add-opens` > > and `--add-exports` command-line options and, recently, the temporary > > but more powerful (and verbose) `--permit-illegal-access` option. > > > > There are no API equivalents to these command-line options [4]. That's > > intentional: We want developers and deployers using Java out-of-the-box > > to be assured that their code is only using APIs actually intended for > > external use. They can choose explicitly to expose internal APIs, via > > the command line, but then that's a deliberate choice that they make, and > > they own the consequences. It's reasonable to allow this via the command > > line since we assume that anyone with access to the command line already > > has the power to corrupt the JDK in any way they please. > > > > The proposal to disable the loading of dynamic agents by default is one > > more part of the overall integrity story. As things stand today, code in > > any JAR file can use the `VirtualMachine.loadAgent` API to load an agent > > into the JVM in which it's running and, via that agent, break into any > > module it likes. Changing the default and providing a new command-line > > option, in this case `-XX:+EnableDyanmicAgentLoading`, is consistent > with > > the other encapsulation-busting options we've introduced. It allows a > > developer or deployer to choose to bend, if not violate, the integrity > > of the platform, yet also to know that by default the platform is whole. > > (Changing this default is also consistent, as Alan has noted, with the > > original intent of the `VirtualMachine.loadAgent` API.) > > > > I understand your points about the practical difficulties of having to > > educate users about this new option and enhance startup scripts to use > > the option only when invoking JDK 9. Isn't it already the ca
Re: Disallowing the dynamic loading of agents by default
On 30/03/2017 20:47, Gregg Wonderly wrote: Mark, what is the plan for people with .jar files on their desktop who just double click on them to run desktop applications? How will they know that their upgrade to Java 9 broke the application and how to remedy the problem by either getting a java-9 compatible version of the app (which will likely not be available at Java-9 release), or that they need to roll back to some previous version of Java until the application they are using is compatible with Java-9? It seems like there is an awful lot of focus on command line solutions when there is a whole class of users who do not use the command line. I can't tell if you meant to reply to this thread or another thread but I suspect you are asking about #AddExportsInManifest [1]. -Alan [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#AddExportsInManifest
Re: Disallowing the dynamic loading of agents by default
I am asking what the Java-9 JVM will do when it encounters any of the errors that the command line arguments are meant to help resolve (temporarily, until the correct module details are created) when run without those command line arguments and the user has a Swing or JavaFX environment active and is not looking at a command line environment. How the problem is fixed from the developer’s perspective is immaterial to my question. I am asking about Java users who install Java-9 (because its out, and they are on Java 8, or some other impetus). How will those users know that they may break applications that they are using, and how will those users resolve the problem themselves? Gregg > On Mar 30, 2017, at 4:30 PM, Alan Bateman wrote: > > On 30/03/2017 20:47, Gregg Wonderly wrote: > >> Mark, what is the plan for people with .jar files on their desktop who just >> double click on them to run desktop applications? How will they know that >> their upgrade to Java 9 broke the application and how to remedy the problem >> by either getting a java-9 compatible version of the app (which will likely >> not be available at Java-9 release), or that they need to roll back to some >> previous version of Java until the application they are using is compatible >> with Java-9? >> >> It seems like there is an awful lot of focus on command line solutions when >> there is a whole class of users who do not use the command line. >> > I can't tell if you meant to reply to this thread or another thread but I > suspect you are asking about #AddExportsInManifest [1]. > > -Alan > > [1] http://openjdk.java.net/projects/jigsaw/spec/issues/#AddExportsInManifest