Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On Nov 1, 2016, at 12:02 PM, Andrew Dinn wrote: > > I did actually suggest a way of avoiding the use of Unsafe. You give > your nominated module full reflective access to java.lang.invoke > allowing it to create Lookup instances (it can actually just create a > single all privileges lookup and use this to clone others). You don't > need to insert a class into java.lang.invoke to do this. You simply add > one exports directive on the command line. The weak link in this otherwise robust scheme is the use of a command line option to break into the jli package. Breaking in by the back door is awkward and might not deliver the desired Lookup if jli code changes. And it probably will, over time. Using the break-in is a good Proof of concept but the finished product needs to use a real API provided by the JDK, and using a wormhole Lookup actually provided by a cooperating jli. I look forward to such a thing in a future JDK. Perhaps the right surface shape for moderate reflection is Maurizios "reflection manifesto" API, which is completely interface driven. This means a meta-CE with full deep access could mock up a "reflection manifesto" implementation which would expose exactly the right moderate surface area, as negotiated with the target module. Just an idea… – John
Re: Request Review: JDK-6479237 (cl) Add support for classloader names
Hi Daniel, Here is the updated webrev incorporating your feedback: http://cr.openjdk.java.net/~mchung/jdk9/webrevs/6479237/webrev.03 > On Nov 1, 2016, at 7:04 AM, Daniel Fuchs wrote: > > : > 334 s += declaringClass; > > > but should line 334 instead be > > s = (s.isEmpty() ? declaringClass : s + "/" + declaringClass; > > Otherwise "/" will be missing after module@version.. > Good catch. I added a test for this. > Also should there be some asserts somewhere verifying that moduleVersion > is null or empty when moduleName is null or empty? At the moment the > constructor will blindly accept a version for an unnamed module, > and I am assuming this is wrong - am I right? > Good point. The constructor should throw IAE if module name/version and class loader name is an empty string. > toLoaderModuleClassName will call ClassLoader.getName(). > If a class loader overrides getName() then that might be a different > name than what StackTraceElement.getClassLoaderName() returns. > > Is that an issue? > I added a test to verify StackTraceElement that uses the ClassLoader's name field. I also added @apiNote in ClassLoader::getName. > Also I wonder whether you should be considering including a fix > for https://bugs.openjdk.java.net/browse/JDK-8167099 as part > of this change (though arguably the bug has been there since > new fields were added to StackTraceElement in 9). > It is related but it’s better to separate it from this issue. Do you have cycle to help write a test case? I can help fix JDK-8167099 later. Mandy
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
Iteration 3, https://gist.github.com/forax/c08ce0a8dc88705bce97b17b63e7e2d5 GrantAccess can now be put on a class, on a package or on a module, and give access to several frameworks by their names. Getting access to the lookup of a class is now a 2 steps process, first claim a framework name to get an AccessFacade and then through the AccessFacade, a framework have access to all classes annotated (directly or indirectly) by GrantAccess and allowing the framework. The facade corresponding to a framework name can only be requested once so how to store the FacadeAccess is let to the framework implementation and if a code try to sneak in and use a framework name it will be detected. Rémi - Mail original - > De: "David M. Lloyd" > À: "Andrew Dinn" , jigsaw-dev@openjdk.java.net > Envoyé: Mardi 1 Novembre 2016 21:00:35 > Objet: Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules > & open packages > On 11/01/2016 11:02 AM, Andrew Dinn wrote: >> On 01/11/16 15:35, David M. Lloyd wrote: >>> On 11/01/2016 10:09 AM, Andrew Dinn wrote: There is a very easy way to provide tightly controlled access to a framework. Export access to e.g. jdk.internal.misc.Unsafe or e.g. java.lang.[invoke].MethodHandles to a nominated module provided by your framework then arrange for that module to hand out Lookups (or better MethodHandles) to framework code as and when they are needed via a private channel. You can do arrange that with a single addExports option on the command line and a very small amount of setup code to establish the private channel >>> >>> I don't see how this is safer though. You're proposing to hand all the >>> keys (or none) to modules which don't need all the keys, whereas I >>> propose that a module should only gain incremental additional access on >>> a grant basis. >> >> No, I'm suggesting handing the keys to a single module provided by your >> framework -- which module will then use a key to open a room on demand >> for your code and your code only. That means the privilege can be >> granted using one command line switch with use of that privilege >> controlled by a runtime mechanism of your choosing. > > Yes, however that means that I'm on the hook to make sure that the keys > aren't mislaid. And given the fact of open modules - which already > semantically grants me privileges - requiring a special command line > switch (which is itself undesirable) and custom code to enable those > privileges in the *preferred* manner (i.e. MethodHandles) seems > backwards when open modules would already grant me the ability to > utilize a much finer instrument albeit with the legacy approach (i.e. > reflection). It seems quite logical to extend this mechanism to the > preferred MethodHandle approach instead, and forget about Unsafe and > anything like it, which I think is in everyone's best interests in the > long term. > >>> At least Rémi's approach requires a specific grant and non-Unsafe >>> vector, though as I said the weaknesses (of his first approach) are that >>> the grant must come from the target class (programmatically) instead of >>> being a static declaration ("opens" is a natural fit for this if I >>> understand the proposal correctly). The other weakness is that it >>> requires class init, but that's purely a consequence of requiring a >>> programmatic grant as far as I understand. >> >> I did actually suggest a way of avoiding the use of Unsafe. You give >> your nominated module full reflective access to java.lang.invoke >> allowing it to create Lookup instances (it can actually just create a >> single all privileges lookup and use this to clone others). You don't >> need to insert a class into java.lang.invoke to do this. You simply add >> one exports directive on the command line. > > That is marginally better, of course, but I think the rest of my > objections still stand against this argument. > >>> Rémi's newest Gist uses an annotation, which is closer but not quite on >>> the target IMO. An annotation cannot be module-deployment-agnostic in >>> the way that I outlined previously (hence >>> #IndirectQualifiedReflectiveAccess); ultimately accessibility >>> information ought to come from the module configuration itself. >> >> If you implement a module with the ability to hand out Lookup instances >> you can use whatever control model you like. For example, you might park >> an instance which hands out Lookups in a location only available to your >> framework or ensure that any caller asking for a Lookup belongs in a >> package implemented by your framework or whatever you want. the point is >> that once you have granted a class in our code the ability to create >> Lookups you can implement whatever runtime access control you want. >> Clearly,as John said, you need to do that responsibly. >> >> regards, >> >> >> Andrew Dinn >> --- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in England a
Re: RFR: 8156499 Update jlink to support creating images with modules that are packaged as multi-release JARs
> On Nov 1, 2016, at 2:44 PM, Steve Drach wrote: > > I’ve put out another webrev, http://cr.openjdk.java.net/~sdrach/8156499/webrev.05/, that addresses Mandy’s concerns. In particular I demonstrate that the resultant image is “runnable” and that a Main class in the image can/cannot find the java.logging module when the module-info.class is changed to require java.logging in one case but not in the other. >>> >>> Looks okay in general. The jlink change looks correct. Thanks for >>> updating the test. >> >> In fact - does this test run on windows? >> >> 239 Path java = Paths.get(image, "bin", "java”); >> >> On windows, shouldn’t this need to be “java.exe”? > > It turns out it runs just fine on the jprt windows test. Thanks. So good to go. Mandy
Re: RFR: 8156499 Update jlink to support creating images with modules that are packaged as multi-release JARs
>>> I’ve put out another webrev, >>> http://cr.openjdk.java.net/~sdrach/8156499/webrev.05/, that addresses >>> Mandy’s concerns. In particular I demonstrate that the resultant image is >>> “runnable” and that a Main class in the image can/cannot find the >>> java.logging module when the module-info.class is changed to require >>> java.logging in one case but not in the other. >> >> Looks okay in general. The jlink change looks correct. Thanks for updating >> the test. > > In fact - does this test run on windows? > > 239 Path java = Paths.get(image, "bin", "java”); > > On windows, shouldn’t this need to be “java.exe”? It turns out it runs just fine on the jprt windows test.
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 11/01/2016 11:02 AM, Andrew Dinn wrote: On 01/11/16 15:35, David M. Lloyd wrote: On 11/01/2016 10:09 AM, Andrew Dinn wrote: There is a very easy way to provide tightly controlled access to a framework. Export access to e.g. jdk.internal.misc.Unsafe or e.g. java.lang.[invoke].MethodHandles to a nominated module provided by your framework then arrange for that module to hand out Lookups (or better MethodHandles) to framework code as and when they are needed via a private channel. You can do arrange that with a single addExports option on the command line and a very small amount of setup code to establish the private channel I don't see how this is safer though. You're proposing to hand all the keys (or none) to modules which don't need all the keys, whereas I propose that a module should only gain incremental additional access on a grant basis. No, I'm suggesting handing the keys to a single module provided by your framework -- which module will then use a key to open a room on demand for your code and your code only. That means the privilege can be granted using one command line switch with use of that privilege controlled by a runtime mechanism of your choosing. Yes, however that means that I'm on the hook to make sure that the keys aren't mislaid. And given the fact of open modules - which already semantically grants me privileges - requiring a special command line switch (which is itself undesirable) and custom code to enable those privileges in the *preferred* manner (i.e. MethodHandles) seems backwards when open modules would already grant me the ability to utilize a much finer instrument albeit with the legacy approach (i.e. reflection). It seems quite logical to extend this mechanism to the preferred MethodHandle approach instead, and forget about Unsafe and anything like it, which I think is in everyone's best interests in the long term. At least Rémi's approach requires a specific grant and non-Unsafe vector, though as I said the weaknesses (of his first approach) are that the grant must come from the target class (programmatically) instead of being a static declaration ("opens" is a natural fit for this if I understand the proposal correctly). The other weakness is that it requires class init, but that's purely a consequence of requiring a programmatic grant as far as I understand. I did actually suggest a way of avoiding the use of Unsafe. You give your nominated module full reflective access to java.lang.invoke allowing it to create Lookup instances (it can actually just create a single all privileges lookup and use this to clone others). You don't need to insert a class into java.lang.invoke to do this. You simply add one exports directive on the command line. That is marginally better, of course, but I think the rest of my objections still stand against this argument. Rémi's newest Gist uses an annotation, which is closer but not quite on the target IMO. An annotation cannot be module-deployment-agnostic in the way that I outlined previously (hence #IndirectQualifiedReflectiveAccess); ultimately accessibility information ought to come from the module configuration itself. If you implement a module with the ability to hand out Lookup instances you can use whatever control model you like. For example, you might park an instance which hands out Lookups in a location only available to your framework or ensure that any caller asking for a Lookup belongs in a package implemented by your framework or whatever you want. the point is that once you have granted a class in our code the ability to create Lookups you can implement whatever runtime access control you want. Clearly,as John said, you need to do that responsibly. regards, Andrew Dinn --- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander -- - DML
JDK 9 Early Access with Project Jigsaw, build 142 on 10-31-2016 (#5670)
Just a quick mail to say that the EA download [1] has been refreshed. The latest download has the initial implementation of open modules and open packages as detailed in the recent proposal for #ReflectiveAccessToNonExportedTypes [2]. -Alan [1] https://jdk9.java.net/jigsaw/ [2] http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-October/000430.html
Re: RFR: 8159523. Fix tests depending on absence of -limitmods in VM arguments.
Alan, For me, having module and package parameters separate makes it a lot easier: public JavaTask addExports(String module, String package, String… targetModules) Consider this: .addExports(JAVA_BASE, JDK_MISC, ALL_UNNAMED) where, needless to say, JAVA_BASE and JDK_MISC are constants defined in the test, or, perhaps, library class and presumably used in other cases. To me, this is better for obvious reasons, such as code completion in IDE and compile-time control over the mistakes. I also will be able to avoid concatenation all over the code: .addExports(JAVA_BASE + “/“ + JDK_MISC, ALL_UNNAMED) Similarly, for the same reasons, I prefer .addExports(JAVA_BASE, JDK_MISC, M1, M2) What do others think? To get everybody on the same page, we are discussing this webrev, which is introducing new task builder API to the JDK test library: http://cr.openjdk.java.net/~shurailine/8159523/webrev.04 This is the review request: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-October/009627.html Shura > On Nov 1, 2016, at 3:26 AM, Alan Bateman wrote: > > > > On 01/11/2016 10:23, Alan Bateman wrote: >> : >> >> .addExports("java.base", "jdk.internal.misc=ALL-UNNAMED") >> .addOpens("java.base", "jdk.internal.misc=m1,m2") > Oops, a typo here, I meant this of course: > > .addExports("java.base/jdk.internal.misc", "ALL-UNNAMED") > .addOpens("java.base/jdk.internal.misc", m1,m2") > > -Alan.
Re: RFR: 8156499 Update jlink to support creating images with modules that are packaged as multi-release JARs
>>> I’ve put out another webrev, >>> http://cr.openjdk.java.net/~sdrach/8156499/webrev.05/, that addresses >>> Mandy’s concerns. In particular I demonstrate that the resultant image is >>> “runnable” and that a Main class in the image can/cannot find the >>> java.logging module when the module-info.class is changed to require >>> java.logging in one case but not in the other. >> >> Looks okay in general. The jlink change looks correct. Thanks for updating >> the test. > > In fact - does this test run on windows? > > 239 Path java = Paths.get(image, "bin", "java”); > > On windows, shouldn’t this need to be “java.exe”? I haven’t run it on jprt yet, but I’ll note that tools/jlink/plugins/SystemModuleDescriptors/UserModuleTest does the same thing (finds java, not java.exe) and has apparently been run on windows many times. If jprt turns up an error here, I’ll fix it.
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 01/11/16 15:35, David M. Lloyd wrote: > On 11/01/2016 10:09 AM, Andrew Dinn wrote: >> There is a very easy way to provide tightly controlled access to a >> framework. Export access to e.g. jdk.internal.misc.Unsafe or e.g. >> java.lang.[invoke].MethodHandles to a nominated module provided by your >> framework then arrange for that module to hand out Lookups (or better >> MethodHandles) to framework code as and when they are needed via a >> private channel. You can do arrange that with a single addExports option >> on the command line and a very small amount of setup code to establish >> the private channel > > I don't see how this is safer though. You're proposing to hand all the > keys (or none) to modules which don't need all the keys, whereas I > propose that a module should only gain incremental additional access on > a grant basis. No, I'm suggesting handing the keys to a single module provided by your framework -- which module will then use a key to open a room on demand for your code and your code only. That means the privilege can be granted using one command line switch with use of that privilege controlled by a runtime mechanism of your choosing. > At least Rémi's approach requires a specific grant and non-Unsafe > vector, though as I said the weaknesses (of his first approach) are that > the grant must come from the target class (programmatically) instead of > being a static declaration ("opens" is a natural fit for this if I > understand the proposal correctly). The other weakness is that it > requires class init, but that's purely a consequence of requiring a > programmatic grant as far as I understand. I did actually suggest a way of avoiding the use of Unsafe. You give your nominated module full reflective access to java.lang.invoke allowing it to create Lookup instances (it can actually just create a single all privileges lookup and use this to clone others). You don't need to insert a class into java.lang.invoke to do this. You simply add one exports directive on the command line. > Rémi's newest Gist uses an annotation, which is closer but not quite on > the target IMO. An annotation cannot be module-deployment-agnostic in > the way that I outlined previously (hence > #IndirectQualifiedReflectiveAccess); ultimately accessibility > information ought to come from the module configuration itself. If you implement a module with the ability to hand out Lookup instances you can use whatever control model you like. For example, you might park an instance which hands out Lookups in a location only available to your framework or ensure that any caller asking for a Lookup belongs in a package implemented by your framework or whatever you want. the point is that once you have granted a class in our code the ability to create Lookups you can implement whatever runtime access control you want. Clearly,as John said, you need to do that responsibly. regards, Andrew Dinn --- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 01.11.2016 16:44, Alan Bateman wrote: On 01/11/2016 15:32, Jochen Theodorou wrote: : Well... it makes me ask the question: Does #AwkwardStrongEncapsulation impose additional limitations for the redefinition of already loaded classes? No but if the methods in the new class redefinition are doing setAccessible then it's same checks as would happen if the original class bytes did the same thing. If I got Remi's proposal right, I would "only" have to add the annotation on the class by instrumentation to gain access, no need for setAccessible bye Jochen
Re: How to check for an internal class in another module from java.base?
On 01/11/2016 15:29, Wang Weijun wrote: : There is an issue in JIRA to track change the HTTP protocol handler to use services for authentication mechanism, I don't think anyone has had time to work on that yet. So before making this big change, do you think I can make the small change above to make them at least start working? i forgot i also exported the internal package to java.base. that is ugly. If this code is changed to use services then the qualified export would not be needed. -Alan.
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 01/11/2016 15:32, Jochen Theodorou wrote: : Well... it makes me ask the question: Does #AwkwardStrongEncapsulation impose additional limitations for the redefinition of already loaded classes? No but if the methods in the new class redefinition are doing setAccessible then it's same checks as would happen if the original class bytes did the same thing. -Alan
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 11/01/2016 10:09 AM, Andrew Dinn wrote: On 01/11/16 14:39, David M. Lloyd wrote: On 11/01/2016 09:23 AM, John Rose wrote: On Nov 1, 2016, at 10:22 AM, Jochen Theodorou wrote: Can we clarify "privileged code"? Privileged like in a SecurityManager in a PrivilegedAction for example, for privileged like only jdk internal code? Just to see it black on white ;) Good question: I mean the basic JDK platform implementation. Something deep in java.base. Like Unsafe. I don't see why this can't be a "regular" API though, rather than a super-user sledgehammer every single time. If user code can be statically granted access, and that user code can deliberately acquire a narrowly-scoped object which can access those Lookups/*Handles, then isn't that better than using Unsafe, which not only represents unrestricted system-wide access, but can undermine even the JVM's integrity if leaked? If making this a 'regular' API means allowing any code to use it then that clearly undermines /all/ the access restrictions provided by modules. The point is not to unlock all doors in the castle but to open doors for trusted code when needed (and only then). Maybe I wasn't clear; I was suggesting that any code be able to use the API, but only to access things which were explicitly granted. I'm suggesting *less* access, not *more*. Saying "suitably privileged" and "Unsafe" seems sufficient until you realize that many many frameworks will need this API, which means they will need Unsafe, which means they will *get* Unsafe (one way or another), which is certainly a far more dangerous thing to have floating around than (say) a Lookup that has access to a few specific private members. There is a very easy way to provide tightly controlled access to a framework. Export access to e.g. jdk.internal.misc.Unsafe or e.g. java.lang.[invoke].MethodHandles to a nominated module provided by your framework then arrange for that module to hand out Lookups (or better MethodHandles) to framework code as and when they are needed via a private channel. You can do arrange that with a single addExports option on the command line and a very small amount of setup code to establish the private channel I don't see how this is safer though. You're proposing to hand all the keys (or none) to modules which don't need all the keys, whereas I propose that a module should only gain incremental additional access on a grant basis. At least Rémi's approach requires a specific grant and non-Unsafe vector, though as I said the weaknesses (of his first approach) are that the grant must come from the target class (programmatically) instead of being a static declaration ("opens" is a natural fit for this if I understand the proposal correctly). The other weakness is that it requires class init, but that's purely a consequence of requiring a programmatic grant as far as I understand. Rémi's newest Gist uses an annotation, which is closer but not quite on the target IMO. An annotation cannot be module-deployment-agnostic in the way that I outlined previously (hence #IndirectQualifiedReflectiveAccess); ultimately accessibility information ought to come from the module configuration itself. -- - DML
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
I must say I am partially to the whole story. My main problem is code I do not have under control, but suddenly need "extended" rights for. If this comes from a another module I will have by default no chance of doing this anymore because of #AwkwardStrongEncapsulation. This proposal shows a way around it, but only if I instrument or otherwise transform the bytecode to add the annotation. If I have that level of control I can do all sorts of things. My simple problem here is, that this may work for a framework you program against, but not for a library that happens to have to process the class without the class knowing anything about that library. And normally you do not want to have a bytecode/class transformer, just to use the library properly. Not to mention, that the point of time in which you get a hold on the object, it might be much too late to do any modifications to the class Well... it makes me ask the question: Does #AwkwardStrongEncapsulation impose additional limitations for the redefinition of already loaded classes? bye Jochen
Re: How to check for an internal class in another module from java.base?
> On Nov 1, 2016, at 11:18 PM, Wang Weijun wrote: > > >> On Nov 1, 2016, at 11:10 PM, Alan Bateman wrote: >> >> >> >> On 01/11/2016 15:02, Wang Weijun wrote: >>> HTTP Negotiate uses >>> >>> Class.forName("sun.net.www.protocol.http.spnego.NegotiatorImpl", true, >>> null); >>> >>> to check if the implementation in java.security.jgss module is provided. It >>> looks like this call is throwing a ClassNotFoundException now. Shall I >>> export the package to java.base or is there any other trick? I remember >>> there used to be an addReads method but seems no one is using it now. >>> >>> Or I must use a service loader? >>> >> This is just visibility. The jdk.security.jgss module is defined to the >> platform class loader but but you have specified "null" (= bootstrap class >> loader) to Class.forName. > > Wow. I changed it to ClassLoaders.platformClassLoader() and it runs. > >> >> There is an issue in JIRA to track change the HTTP protocol handler to use >> services for authentication mechanism, I don't think anyone has had time to >> work on that yet. > > So before making this big change, do you think I can make the small change > above to make them at least start working? i forgot i also exported the internal package to java.base. that is ugly. thanks max p.s. have to use only hand so the other hand can grab my daughter. > > Thanks > Max > >> >> -Alan. >
Re: How to check for an internal class in another module from java.base?
> On Nov 1, 2016, at 11:10 PM, Alan Bateman wrote: > > > > On 01/11/2016 15:02, Wang Weijun wrote: >> HTTP Negotiate uses >> >> Class.forName("sun.net.www.protocol.http.spnego.NegotiatorImpl", true, >> null); >> >> to check if the implementation in java.security.jgss module is provided. It >> looks like this call is throwing a ClassNotFoundException now. Shall I >> export the package to java.base or is there any other trick? I remember >> there used to be an addReads method but seems no one is using it now. >> >> Or I must use a service loader? >> > This is just visibility. The jdk.security.jgss module is defined to the > platform class loader but but you have specified "null" (= bootstrap class > loader) to Class.forName. Wow. I changed it to ClassLoaders.platformClassLoader() and it runs. > > There is an issue in JIRA to track change the HTTP protocol handler to use > services for authentication mechanism, I don't think anyone has had time to > work on that yet. So before making this big change, do you think I can make the small change above to make them at least start working? Thanks Max > > -Alan.
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
- Mail original - > De: "David M. Lloyd" > À: jigsaw-dev@openjdk.java.net > Envoyé: Mardi 1 Novembre 2016 15:39:01 > Objet: Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules > & open packages > On 11/01/2016 09:23 AM, John Rose wrote: >> On Nov 1, 2016, at 10:22 AM, Jochen Theodorou wrote: >>> >>> Can we clarify "privileged code"? Privileged like in a SecurityManager in a >>> PrivilegedAction for example, for privileged like only jdk internal code? >>> Just >>> to see it black on white ;) >> >> Good question: I mean the basic JDK platform implementation. Something >> deep in >> java.base. Like Unsafe. > > I don't see why this can't be a "regular" API though, rather than a > super-user sledgehammer every single time. If user code can be > statically granted access, and that user code can deliberately acquire a > narrowly-scoped object which can access those Lookups/*Handles, then > isn't that better than using Unsafe, which not only represents > unrestricted system-wide access, but can undermine even the JVM's > integrity if leaked? > It's better than Unsafe because as a user you have to grant access by using by example an annotation, and you can specifies friends and/or what you want to export (only private/only package private, etc). But if the API returns a Lookup object and a client code with granted access exposes that lookup, all bets are off. > -- > - DML Rémi
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
Oops, wrong package! -- corrected inline On 01/11/16 15:09, Andrew Dinn wrote: > On 01/11/16 14:39, David M. Lloyd wrote: >> On 11/01/2016 09:23 AM, John Rose wrote: >>> On Nov 1, 2016, at 10:22 AM, Jochen Theodorou wrote: Can we clarify "privileged code"? Privileged like in a SecurityManager in a PrivilegedAction for example, for privileged like only jdk internal code? Just to see it black on white ;) >>> >>> Good question: I mean the basic JDK platform implementation. >>> Something deep in java.base. Like Unsafe. >> >> I don't see why this can't be a "regular" API though, rather than a >> super-user sledgehammer every single time. If user code can be >> statically granted access, and that user code can deliberately acquire a >> narrowly-scoped object which can access those Lookups/*Handles, then >> isn't that better than using Unsafe, which not only represents >> unrestricted system-wide access, but can undermine even the JVM's >> integrity if leaked? > > If making this a 'regular' API means allowing any code to use it then > that clearly undermines /all/ the access restrictions provided by > modules. The point is not to unlock all doors in the castle but to open > doors for trusted code when needed (and only then). > > There is a very easy way to provide tightly controlled access to a > framework. Export access to e.g. jdk.internal.misc.Unsafe or e.g. > java.lang.reflect.MethodHandles to a nominated module provided by your ^ +--java.lang.invoke.MethodHandles > framework then arrange for that module to hand out Lookups (or better > MethodHandles) to framework code as and when they are needed via a > private channel. You can do arrange that with a single addExports option > on the command line and a very small amount of setup code to establish > the private channel > > regards, > > > Andrew Dinn > --- > > -- regards, Andrew Dinn --- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
Re: How to check for an internal class in another module from java.base?
On 01/11/16 15:10, Alan Bateman wrote: There is an issue in JIRA to track change the HTTP protocol handler to use services for authentication mechanism, I don't think anyone has had time to work on that yet. This is https://bugs.openjdk.java.net/browse/JDK-8038079 -- daniel
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
- Mail original - > De: "John Rose" > À: "David M. Lloyd" > Cc: jigsaw-dev@openjdk.java.net > Envoyé: Mardi 1 Novembre 2016 15:04:33 > Objet: Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules > & open packages > On Nov 1, 2016, at 9:53 AM, David M. Lloyd wrote: >> >> 1. It requires the target class to be initialized >> 2. It requires the target class to proactively donate MethodHandles or a >> Lookup >> to the lookup class > > Both of these can be overcome, though only by privileged code. > The privileged code would forge (uh, "mint") a legitimate Lookup to the > not-yet-initialized class. > A "Vault" meta-framework doesn't need to inject a Lookup donation statement > into > anybody's . > Instead, it needs to do the super-user operation of making a trusted lookup. > It must also fulfill the super-user *responsibility* of not leaking such > lookups, just using them in a predictable, rule-based manner. > — John yes, being in java.lang.invoke is enough, https://gist.github.com/forax/cab12206323ca2e5866c9f8e20ee6d40 Rémi
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 01/11/16 14:39, David M. Lloyd wrote: > On 11/01/2016 09:23 AM, John Rose wrote: >> On Nov 1, 2016, at 10:22 AM, Jochen Theodorou wrote: >>> >>> Can we clarify "privileged code"? Privileged like in a >>> SecurityManager in a PrivilegedAction for example, for privileged >>> like only jdk internal code? Just to see it black on white ;) >> >> Good question: I mean the basic JDK platform implementation. >> Something deep in java.base. Like Unsafe. > > I don't see why this can't be a "regular" API though, rather than a > super-user sledgehammer every single time. If user code can be > statically granted access, and that user code can deliberately acquire a > narrowly-scoped object which can access those Lookups/*Handles, then > isn't that better than using Unsafe, which not only represents > unrestricted system-wide access, but can undermine even the JVM's > integrity if leaked? If making this a 'regular' API means allowing any code to use it then that clearly undermines /all/ the access restrictions provided by modules. The point is not to unlock all doors in the castle but to open doors for trusted code when needed (and only then). There is a very easy way to provide tightly controlled access to a framework. Export access to e.g. jdk.internal.misc.Unsafe or e.g. java.lang.reflect.MethodHandles to a nominated module provided by your framework then arrange for that module to hand out Lookups (or better MethodHandles) to framework code as and when they are needed via a private channel. You can do arrange that with a single addExports option on the command line and a very small amount of setup code to establish the private channel regards, Andrew Dinn ---
Re: How to check for an internal class in another module from java.base?
On 01/11/2016 15:02, Wang Weijun wrote: HTTP Negotiate uses Class.forName("sun.net.www.protocol.http.spnego.NegotiatorImpl", true, null); to check if the implementation in java.security.jgss module is provided. It looks like this call is throwing a ClassNotFoundException now. Shall I export the package to java.base or is there any other trick? I remember there used to be an addReads method but seems no one is using it now. Or I must use a service loader? This is just visibility. The jdk.security.jgss module is defined to the platform class loader but but you have specified "null" (= bootstrap class loader) to Class.forName. There is an issue in JIRA to track change the HTTP protocol handler to use services for authentication mechanism, I don't think anyone has had time to work on that yet. -Alan.
How to check for an internal class in another module from java.base?
HTTP Negotiate uses Class.forName("sun.net.www.protocol.http.spnego.NegotiatorImpl", true, null); to check if the implementation in java.security.jgss module is provided. It looks like this call is throwing a ClassNotFoundException now. Shall I export the package to java.base or is there any other trick? I remember there used to be an addReads method but seems no one is using it now. Or I must use a service loader? Thanks Max
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 01/11/16 14:23, John Rose wrote: > On Nov 1, 2016, at 10:22 AM, Jochen Theodorou > wrote: >> >> Can we clarify "privileged code"? Privileged like in a >> SecurityManager in a PrivilegedAction for example, for privileged >> like only jdk internal code? Just to see it black on white ;) > > Good question: I mean the basic JDK platform implementation. > Something deep in java.base. Like Unsafe. Or indeed you might only require enough privilege to call the public MethodHandles.Lookup constructor -- n.b. although that constructor is public its package is not exported to the default module. Of course, you can always arrange for it to be exported to one of /your/ modules. Note that whatever mechanism is used to obtain Lookup instances this has several advantages. 1) Once a privileged module is able to create Lookups (with suitably constrained access) arranging for it to safely hand them over to your framework code (and to no other code) when needed and with no more access than is actually merited is relatively simple. Better still arrange to hand over the requisite MethodHandles and keep the Lookup instances secured. Whereas If you want to use reflection safely then you have to be able to call setAccessible from a module which is an export target for the package of the class whose member you want to access. In general you don't want to export all possible reflection targets to all your framework code (especially when it is in the default module). So, you end up having to set up some privileged module and arrange for it call setAccessible on behalf of your framework code (and no other code). Configuring all potential exports to a privileged module in advance on the command line will be tricky/verbose at best to and may well not even be possible. If you have an agent that is capable of editing exports lists then you can rewrite module exports to your privileged module on the fly but that is also a lot of work (I know because I have tried it). 2) A MethodHandle created by a Lookup is not opaque to the compiler. So this creates opportunities for the compiler to optimize the code that employs them. Accessible fields/methods/constructors are opaque to the compiler which makes it much less likely that optimization will be possible. regards, Andrew Dinn ---
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 11/01/2016 09:23 AM, John Rose wrote: On Nov 1, 2016, at 10:22 AM, Jochen Theodorou wrote: Can we clarify "privileged code"? Privileged like in a SecurityManager in a PrivilegedAction for example, for privileged like only jdk internal code? Just to see it black on white ;) Good question: I mean the basic JDK platform implementation. Something deep in java.base. Like Unsafe. I don't see why this can't be a "regular" API though, rather than a super-user sledgehammer every single time. If user code can be statically granted access, and that user code can deliberately acquire a narrowly-scoped object which can access those Lookups/*Handles, then isn't that better than using Unsafe, which not only represents unrestricted system-wide access, but can undermine even the JVM's integrity if leaked? -- - DML
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On Nov 1, 2016, at 10:22 AM, Jochen Theodorou wrote: > > Can we clarify "privileged code"? Privileged like in a SecurityManager in a > PrivilegedAction for example, for privileged like only jdk internal code? > Just to see it black on white ;) Good question: I mean the basic JDK platform implementation. Something deep in java.base. Like Unsafe.
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On 01.11.2016 15:04, John Rose wrote: On Nov 1, 2016, at 9:53 AM, David M. Lloyd wrote: 1. It requires the target class to be initialized 2. It requires the target class to proactively donate MethodHandles or a Lookup to the lookup class Both of these can be overcome, though only by privileged code. The privileged code would forge (uh, "mint") a legitimate Lookup to the not-yet-initialized class. A "Vault" meta-framework doesn't need to inject a Lookup donation statement into anybody's . Instead, it needs to do the super-user operation of making a trusted lookup. It must also fulfill the super-user *responsibility* of not leaking such lookups, just using them in a predictable, rule-based manner. Can we clarify "privileged code"? Privileged like in a SecurityManager in a PrivilegedAction for example, for privileged like only jdk internal code? Just to see it black on white ;) bye Jochen
Re: Request Review: JDK-6479237 (cl) Add support for classloader names
Hi Mandy, I am sure I must be missing something: 322 if (s == null) { 323 // all elements will be included 324 s = ""; 325 if (classLoaderName != null && !classLoaderName.isEmpty()) { 326 s += classLoaderName + "/"; 327 } 328 if (moduleName != null && !moduleName.isEmpty()) { 329 s += moduleName; 330 } 331 if (moduleVersion != null && !moduleVersion.isEmpty()) { 332 s += "@" + moduleVersion; 333 } 334 s += declaringClass; 335 } but should line 334 instead be s = (s.isEmpty() ? declaringClass : s + "/" + declaringClass; Otherwise "/" will be missing after module@version.. Also should there be some asserts somewhere verifying that moduleVersion is null or empty when moduleName is null or empty? At the moment the constructor will blindly accept a version for an unnamed module, and I am assuming this is wrong - am I right? toLoaderModuleClassName will call ClassLoader.getName(). If a class loader overrides getName() then that might be a different name than what StackTraceElement.getClassLoaderName() returns. Is that an issue? Also I wonder whether you should be considering including a fix for https://bugs.openjdk.java.net/browse/JDK-8167099 as part of this change (though arguably the bug has been there since new fields were added to StackTraceElement in 9). best regards, -- daniel On 28/10/16 21:44, Mandy Chung wrote: On Oct 28, 2016, at 11:11 AM, Brent Christian wrote: Should something be done for STEs returned from StackFrameInfo.toStackTraceElement() ? Good catch - I missed it. I added package-private static methods in StackTraceElement class for both Throwable and StackFrameInfo to get StackTraceElement(s). http://cr.openjdk.java.net/~mchung/jdk9/webrevs/6479237/webrev.02/ Mandy
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
On Nov 1, 2016, at 9:53 AM, David M. Lloyd wrote: > > 1. It requires the target class to be initialized > 2. It requires the target class to proactively donate MethodHandles or a > Lookup to the lookup class Both of these can be overcome, though only by privileged code. The privileged code would forge (uh, "mint") a legitimate Lookup to the not-yet-initialized class. A "Vault" meta-framework doesn't need to inject a Lookup donation statement into anybody's . Instead, it needs to do the super-user operation of making a trusted lookup. It must also fulfill the super-user *responsibility* of not leaking such lookups, just using them in a predictable, rule-based manner. — John
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
I just want to point out the two obvious (closely related) major problems with this approach which prevent it from being a practical replacement for setAcceptable, lest they are forgotten/ignored in the excitement around the new ideas: 1. It requires the target class to be initialized 2. It requires the target class to proactively donate MethodHandles or a Lookup to the lookup class Both of these are effectively pretty substantial usability regressions compared to the status quo. These problems are why I'd rather see some kind of static grant mechanism, and either a special static API to get an extra-privileged Lookup or the appropriate magic to allow a class's "normal" Lookup object to "see" everything that was granted to it; but really any idea which would solve both of these problems in any way should be put forward for consideration IMO. On 10/31/2016 06:57 PM, Remi Forax wrote: Hi Paul, I think you can already sketch an implementation of the proposal now by using either an annotation processor + a jlink plugin, or by a jlink plugin + a bytecode processor tool. I believe that you can push the information needed by the meta-framework to fulfill requests by adding some method calls inside the static initializer of the classes that are annotated. The runtime part can be something along that line https://gist.github.com/forax/1511fae2273f04273ff9463c6fbbdfbc Instead of providing method handles as John suggests, i provide the lookup object which is less secure. And the Class object in the map are stored as strong ref instead of being an ephemeron (the lambda used as value in the map has also a strong reference on the class). Rémi - Mail original - De: "Paul Bakker" À: jigsaw-dev@openjdk.java.net Envoyé: Lundi 31 Octobre 2016 18:06:27 Objet: Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages The proposal looks very good! Thanks for listening to the voiced concerns. When can we expect a new prototype containing the proposed features? Thanks, Paul On Fri, Oct 28, 2016 at 3:38 PM John Rose wrote: On Oct 27, 2016, at 9:07 AM, mark.reinh...@oracle.com wrote: Further comments most welcome, as usual! +100 for *qualified* opens; this puts an important limit on deep reflection. My main concern with reflection is to avoid "falling off the encapsulation cliff" the first time a user module wishes to open itself up for deep reflection. By such a "cliff" I mean that when deep reflection is allowed, potentially any name in the module can be inspected. Even of only 0.01% of names are actually inspected, tools for reorganizing code must assume that 100% of the names *might* be inspected, at some point in the future, by the allowed party. So qualified opens is a partial solution. And, I think it is exactly right for today, because it can be extended tomorrow. I.e., it is future-friendly. I'll explain by sketching a concept called "moderate reflection", or "MR". A fuller solution would allow other tools to make conclusions about more specific limitations on the actual extent of the deep reflection. Call such an extent-limited reflection moderate reflection, where "moderation" is declared statically by some sort of flexible declarative rule set. In this way tools could make more detailed conclusions about the encapsulation of particular names in the module. For future JDK releases we can consider layering of this qualified opens, by creating a trusted, parameterized meta-framework ("Moderate Reflection") to which a user module can be qualified-open. This meta-framework can then perform deep reflection on behalf of self-describing "moderately reflecting" client frameworks. User modules would not directly open themselves to any module other than MR. The advantage would come when the client frameworks explicitly declare their self-restraint ("moderation") in reflection. For example, the "Foo" framework might limit itself to names annotated with @FooFrameworkHook. An AOT compiler (or other jlink-time reorganization tool) would (1) note that some user module bar is qualified-open only to Moderate Reflection, and has an export (maybe qualified) to a Foo framework. At runtime, (2) the Foo framework would ask Moderate Reflection to reflect into bar. Moderate Reflection would (3) check the self-limitation on Foo, and also ensure that bar exports (though doesn't open) to Foo. MR would then gain the requested access and delegate it back to Foo. Finally, (4) the AOT compiler would free itself to omit metadata for unexported names not visible to moderate reflection. (Method handles are a good way to grant access, since they can be delegated from MR to Foo without re-authorizing Foo. Core Reflection re-authorizes on every access, which awkwardly requires deep reflection on usage, as well as initial lookup.) Moderate Reflection might also look at metadata on the bar module (besides its module exports) to impose further limits on
hg: jigsaw/jake/jdk: Import of 8168980: Reinstate sun.reflect.ReflectionFactory.newConstructorForSerialization(Class, Constructor)
Changeset: 7992a052c7fa Author:chegar Date: 2016-11-01 12:37 + URL: http://hg.openjdk.java.net/jigsaw/jake/jdk/rev/7992a052c7fa Import of 8168980: Reinstate sun.reflect.ReflectionFactory.newConstructorForSerialization(Class,Constructor) Reviewed-by: alanb ! src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java ! src/jdk.unsupported/share/classes/sun/reflect/ReflectionFactory.java ! test/sun/reflect/ReflectionFactory/ReflectionFactoryTest.java
Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules & open packages
- Mail original - > De: "Remi Forax" > À: "Paul Bakker" > Cc: jigsaw-dev@openjdk.java.net > Envoyé: Mardi 1 Novembre 2016 00:57:47 > Objet: Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open modules > & open packages > Hi Paul, > I think you can already sketch an implementation of the proposal now by using > either an annotation processor + a jlink plugin, or by a jlink plugin + a > bytecode processor tool. > I believe that you can push the information needed by the meta-framework to > fulfill requests by adding some method calls inside the static initializer of > the classes that are annotated. > The runtime part can be something along that line > > https://gist.github.com/forax/1511fae2273f04273ff9463c6fbbdfbc > > Instead of providing method handles as John suggests, i provide the lookup > object which is less secure. > And the Class object in the map are stored as strong ref instead of being an > ephemeron (the lambda used as value in the map has also a strong reference on > the class). After shower edit, i've updated the gist to use a ClassValue + a CAS so there is no strong ref to the classes now. > > Rémi Rémi > > - Mail original - >> De: "Paul Bakker" >> À: jigsaw-dev@openjdk.java.net >> Envoyé: Lundi 31 Octobre 2016 18:06:27 >> Objet: Re: New proposal for #ReflectiveAccessToNonExportedTypes: Open >> modules & >> open packages > >> The proposal looks very good! Thanks for listening to the voiced concerns. >> When can we expect a new prototype containing the proposed features? >> >> Thanks, >> >> Paul >> >> On Fri, Oct 28, 2016 at 3:38 PM John Rose wrote: >> >>> On Oct 27, 2016, at 9:07 AM, mark.reinh...@oracle.com wrote: >>> > >>> > Further comments most welcome, as usual! >>> >>> >>> +100 for *qualified* opens; this puts an important limit on deep >>> reflection. >>> >>> My main concern with reflection is to avoid "falling off the encapsulation >>> cliff" >>> the first time a user module wishes to open itself up for deep reflection. >>> By such a "cliff" I mean that when deep reflection is allowed, potentially >>> any name in the module can be inspected. Even of only 0.01% of names >>> are actually inspected, tools for reorganizing code must assume that 100% >>> of the names *might* be inspected, at some point in the future, by the >>> allowed party. >>> >>> So qualified opens is a partial solution. And, I think it is exactly >>> right for >>> today, because it can be extended tomorrow. I.e., it is future-friendly. >>> I'll explain by sketching a concept called "moderate reflection", or "MR". >>> >>> A fuller solution would allow other tools to make conclusions about more >>> specific limitations on the actual extent of the deep reflection. Call >>> such >>> an extent-limited reflection moderate reflection, where "moderation" is >>> declared statically by some sort of flexible declarative rule set. In >>> this way >>> tools could make more detailed conclusions about the encapsulation of >>> particular names in the module. >>> >>> For future JDK releases we can consider layering of this qualified opens, >>> by creating a trusted, parameterized meta-framework ("Moderate Reflection") >>> to which a user module can be qualified-open. This meta-framework can >>> then perform deep reflection on behalf of self-describing "moderately >>> reflecting" >>> client frameworks. User modules would not directly open themselves to any >>> module other than MR. >>> >>> The advantage would come when the client frameworks explicitly declare >>> their >>> self-restraint ("moderation") in reflection. For example, the "Foo" >>> framework >>> might limit itself to names annotated with @FooFrameworkHook. >>> >>> An AOT compiler (or other jlink-time reorganization tool) would (1) note >>> that >>> some user module bar is qualified-open only to Moderate Reflection, and >>> has an export (maybe qualified) to a Foo framework. At runtime, (2) the >>> Foo framework would ask Moderate Reflection to reflect into bar. >>> Moderate Reflection would (3) check the self-limitation on Foo, and also >>> ensure that bar exports (though doesn't open) to Foo. MR would then >>> gain the requested access and delegate it back to Foo. Finally, (4) the >>> AOT compiler would free itself to omit metadata for unexported names >>> not visible to moderate reflection. >>> >>> (Method handles are a good way to grant access, since they can >>> be delegated from MR to Foo without re-authorizing Foo. Core >>> Reflection re-authorizes on every access, which awkwardly >>> requires deep reflection on usage, as well as initial lookup.) >>> >>> Moderate Reflection might also look at metadata on the bar module >>> (besides its module exports) to impose further limits on reflection. >>> Thus, MR-specific metadata might prevent MR from granting bar >>> access to normally-trusted frameworks that bar doesn't trust, if >>> bar can blacklist them for itself in a place that MR checks. >>> >>> And
Re: RFR: 8159523. Fix tests depending on absence of -limitmods in VM arguments.
On 01/11/2016 10:23, Alan Bateman wrote: : .addExports("java.base", "jdk.internal.misc=ALL-UNNAMED") .addOpens("java.base", "jdk.internal.misc=m1,m2") Oops, a typo here, I meant this of course: .addExports("java.base/jdk.internal.misc", "ALL-UNNAMED") .addOpens("java.base/jdk.internal.misc", m1,m2") -Alan.
Re: RFR: 8159523. Fix tests depending on absence of -limitmods in VM arguments.
On 31/10/2016 21:39, Alexandre (Shura) Iline wrote: : Which syntax would be better? 1. Requiring explicit ALL-UNNAMED .addExports("java.base", “jdk.internal.misc”, ALL_UNNAMED) .addExports("java.base", "jdk.internal.reflect”, ALL_UNNAMED) 2. Treating ALL-UNNAMED as a default .addExports("java.base", “jdk.internal.misc”) .addExports("java.base", "jdk.internal.reflect”) Same for addReads(…) and, I assume, addOpens(…). With any of the two, I am in favor of removing addExports(String), addReads(String) and addOpens(String) methods. If a complete string parameter needs to be passed (such as for negative tests), it could be passed through the vmOptions(String …) method. What do you think? All very subjective and I'm sure others will have different opinions. For these three options then the value has two parts, `--add-reads =` for example, and the simplest is to just provide a method that takes two parameters, e.g. : .addReads("m1, "m2") .addExports("java.base", "jdk.internal.misc=ALL-UNNAMED") .addOpens("java.base", "jdk.internal.misc=m1,m2") .build(); The resulting tests should be very readable. Also it has the benefit that we can send bad values, important for many of these tests because they are probing behavior with bad modules, packages, ... For addModules then `String module, String... rest` should work nicely: .addModules("java.corba") .addModules("java.naming", "java.xml.bind") .addModules("jdk.attach,jdk.jdi") -Alan
Re: Request Review: JDK-6479237 (cl) Add support for classloader names
On 31/10/2016 21:34, Mandy Chung wrote: On Oct 31, 2016, at 6:51 AM, Alan Bateman wrote: if (VM.isModuleSystemInited() && !HashedModules.contains(m)) { Looks okay, an alternative would be to move that that check to L439. What do you think about this? http://cr.openjdk.java.net/~mchung/jdk9/webrevs/6479237/webrev.03/ This looks okay as we'll only have frames for java.base during early startup. -Alan