Authorization Layer post JEP 411
A comment from Ron highlites our issue: the JDK contains only things that either only the JDK can technically do We have a need to distinguish between different sources of code, as well as user principles, and as well as Services. Our services are loaded by separate ClassLoaders and are to some extent sandboxed as Ron's example suggests. And we have a need to control access based on these entities. We have always strived to be cross platform and tested on other JVM's such as J9. It's just very hard to see any solutions without AccessController and AccessControlContext. We don't need SecurityManager (although we still need a Policy provider, because ProtectionDomain calls it, but we don't need a policy implementation, just the provider, feel free to remove Java's PolicyFile implementation), if we added a provider interface to Guard.check and changed all permission checks to call their superclass method Guard.check. That authorization layer provider could be called Authority and it can have one single method: Authority::confirm(Permission p) throws SecurityException; We need the power of AccessController's stack walk, StackWalker doesn't work with compiled code, only bytecode. AccessController and AccessControlContext allow backward compatiblity for JAAS. JAAS whether we like it or not, is the default authorisation layer framework. http://word-bits.flurg.com/jaas-is-terrible-and-there-is-no-escape-from-it/ We could create a new property that bypasses the AccessController's stack walk for those who don't need to control CodeSource access. (Just create a ProtectionDomain containing a Subject). Benefits: With SecurityManager gone, people will no longer assume it has sole responsible for Security and OpenJDK devs won't carry a significant burden for it's maintenance. Any security issues will be the responsibility of third party implementations, like mine. The JDK won't provide an implementation, just the framework. Those of us using the Principle of Least Privilege can continue to do so and we can participate in OpenJDK to maintain Permission checks where we need them and preserve context where appropriate. JAAS will continue to remain functional and it's performance will increase significantly (it performs very well with my Policy implementation, even with stack walks). -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: JEP 411 - Secure Java Distribution
Thanks Andrew, I've been thinking about how to do this in a compatible manner. The Guard.check call checks whether SecurityManager is enabled. All permission checks in the JDK could be changed to call the Guard.check method. Unfortunately other permission checks in user code will be broken, if they don't utilise this method, but often the most important permissions are those defined in JDK code. If we modify this method to look for a provider that's outside the java.* namespace. Then we have a point in the code where we can do something like SecurityManager, without requiring SecurityManager. Next step will be to replace all AccessController and AccessControlContext uses in JDK code with another class outside the java.* namespace. In the first instance, we allow SecurityManager, AccessController and AccessControlContext to behave in their degraded state, passing all JCK tests. But then we can create providers to replace their functionality. This is my current line of thought, as other classes will remain. Subject.doAs could be an issue, so would need to consider how to manage that. This seems the easiest way. Then for backward compatibility, we make a tool that rewrites java bytecode, to replace the calls to AccessController and AccessControlContext with compatible equivalents outside the java.* namespace in client code. Then there's the case of removing any references to SecurityManager and looking for SecurityManager permission checks and replacing them with Guard.check. Then we could potentially continue to support this functionality on later versions of Java without detonation. Cheers, Peter. On 2/06/2021 7:35 pm, Andrew Haley wrote: On 6/1/21 10:06 AM, Peter Firmstone wrote: If a vendor were to continue supporting SecurityManager and was backporting from OpenJDK, if it passes the JCK with SecurityManager disabled, that's still acceptable right? Look at the licence agreement in conjunction with the JCK users' guide. See the definition of “Compatible Licensee Implementation”.
JEP 411 - Secure Java Distribution
If a vendor were to continue supporting SecurityManager and was backporting from OpenJDK, if it passes the JCK with SecurityManager disabled, that's still acceptable right? -- Regards, Peter Firmstone
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Would have really liked to have known about that. Any chance Java 17 LTS (hopefully with sorted security manager property), can be supported as long as Java 8, 16 years? The new security API's don't exist yet, I'd prefer to work with a version that has a fully functional SecurityManager rather than one that's non functional, because it's the functionality that's important to us. It's going to take a long time to migrate, once the new API's exist, they need to receive acceptance, then we also need to rewrite our software to suit, assuming it takes 4 years to write and implement the new security api, there's not much time left then before 17's EOL if it only has 8 years of support. I figure there's no harm in asking. Regards, Peter. On 1/06/2021 3:51 pm, Alan Bateman wrote: On 01/06/2021 01:11, Chapman Flack wrote: On 05/31/21 03:59, Alan Bateman wrote: The SM survey in 2018 showed that there was some usage too. Out of curiosity, where can I learn more about this survey? Was it the kind of survey that, if I had been hanging around in the right places in 2018, I might have been solicited to respond to? The survey that has been referenced a few times here was in Feb 2018 [1]. Sean linked to the results at the time [2] but the link no longer works. The main goal was to gather data on usages beyond applets and JNLP. -Alan [1] https://twitter.com/seanjmullan/status/960553938553630721 [2] https://twitter.com/seanjmullan/status/976466563997028354
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Thanks Alan, Concurrency is difficult too, but Java has gone a long way to addressing this with concurrency libraries that make the task seem like child's play, compared to using Java 1.4 language constructs. On 31/05/2021 5:59 pm, Alan Bateman wrote: On 31/05/2021 08:11, Peter Firmstone wrote: : I think also that many more people are using SecurityManager than OpenJDK realises, and they're not using it how OpenJDK recommends either, (AllPermission granted to trusted code, and sandbox untrusted code model of Applets is not how we use it) people are using POLP, it's just that no one reports back to OpenJDK because they are only editing policy files, it will work with any library out there now, there's nothing to write back about. It's pretty clear that OpenJDK devs don't use it, but they do have to manage doPrivileged and preserving context across tasks and threads. And there are static analysis tools in Spotbugs to identify doPrivileged bugs, but someone has recently suggested removing them thanks to this JEP? Does OpenJDK use static analysis. I think if you did you'd find plenty of latent bugs. I don't think the SM is approachable by most developers. I've sat through several embarrassing sessions at conferences over the years where a speaker attempted to get something non-trivial to work with the SM enabled. More often than not they had to deal with libraries that had never been run with a SM before and it was whack-a-mole to get them to run. Yes, it shouldn't have been allowed out the door without a tool for that reason, it could have been avoided. Yes, I have a tool that sorts that very problem, it's 500 - 600 lines of code. Just specify two command line arguments which are the security manager and policy file location, it generates the policy file, then you look at the file briefly, alter some permissions like network for ephemeral ports, easy, you now have a POLP policy file. You do need to properly run the program through its use cases however, but testing is often sufficient, even if you don't have full test coverage it will capture 99.9% of permissions required, then if you tests are missing a permission, it indicates insufficient test coverage. The tool will also append, so you can run multiple processes and user roles one after the other. I will rewrite it in GPL2.0 if you ask me to. The only reason I didn't get around to it before was it's AL2.0 and there wasn't an existential threat to SecurityManager API's. As you know, any library or framework with callbacks means careful use of doPrivileged to avoid needing to grant permissions to every component on the stack. You mention capturing and re-asserting contexts across threads, this is just way too complex for most developers. My guess is that if we had enabled the SM by default 20 years ago then it would be a different discussion today. This is not to say that there isn't some usage, the flurry of mails here over the last month does show that there is some usage. The SM survey in 2018 showed that there was some usage too. Yes, so how do we start that process? It doesn't matter if we start it now or 20 years ago, as long as when it does happen the outcome is positive, I have been coming across more and more developers who implement POLP or that want to. Executors has two methods which decorate Runnable and Callable, but they could be placed into the Executors and performed automatically when tasks are submitted, when SecurityManager is installed. We could give developers a choice, so loom threads are zero privileged by default or are treated as doPrivileged calls with the Subject of the creator, so it only has one ProtectionDomain and load it with immutable collections of permissions, then we can use them for TLS connections if we want or leave them unprivileged. It's a fair compromise, just make sure the callers domain has minimal privileges. It seems to me that loom would be very good for blocking IO. It only needs to be documented, then we will use it appropriately. In my SecurityManager implementation, once an AccessControlContext has been checked for a Permission, it is only ever checked again if the policy is refreshed, so it's a once off which has little impact in an Executor pool. It uses concurrent collections that are decorated with weak references, to avoid memory leaks. I can't speak for all OpenJDK contributors but Oracle contributors do a massive amount of analysis and static analysis before proposing to deprecate or removing anything. It's often much harder to remove something that add it and I don't think anyone has proposed deprecating or removing anything without a strong case. -Alan I understand, but this is incredibly useful software when POLP is applied and if promoted properly with good tooling
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Hmm, whitelisting, you're using the principle of least privilege too then. We have multiple SecurityManager and Policy implementations. Our policy implementations can decorate each other with different functionality, including dynamic permissions and permissions granted by a permission service remotely, they have an interface to bubble up immutable data, so they can share immutable state. The implementation code that ships with Java is like a model T Ford, ours is Formula 1, concurrent and non blocking, it's also very performant, we use bit shift operations, for string normalization, we've got RFC 3986 URI normalization and RFC 5982 IPv6 normalization , Java's implementation code makes network calls to DNS. There just is no comparison there at all. We use a comparator to order the permissions into their most performant order and it doesn't call equals or hashcode as that can cause a DNS call. I believe we do achieve it's theoretical utility and I don't know that I would call it absolutely amazing, but it gets out of the way and just does what's is supposed to reliably, without impacting performance. But now OpenJDK wants to rip the rug out from underneath us, because it's causing problems with loom or some other projects. Could you just make the AccessControlContext immutable for loom threads? We might say, ok I need a few minimal permissions for this AccessControlContext, give it one ProtectionDomain with a Subject and static Permissions, so it doesn't consult the policy, that AccessControlContext is immutable. Get rid of the locks in PermissionCollection and make them immutable with a builder. We create PermissionCollection on demand and discard their reference for the garbage collection, we don't actually share PermissionCollection instances. I think also that many more people are using SecurityManager than OpenJDK realises, and they're not using it how OpenJDK recommends either, (AllPermission granted to trusted code, and sandbox untrusted code model of Applets is not how we use it) people are using POLP, it's just that no one reports back to OpenJDK because they are only editing policy files, it will work with any library out there now, there's nothing to write back about. It's pretty clear that OpenJDK devs don't use it, but they do have to manage doPrivileged and preserving context across tasks and threads. And there are static analysis tools in Spotbugs to identify doPrivileged bugs, but someone has recently suggested removing them thanks to this JEP? Does OpenJDK use static analysis. I think if you did you'd find plenty of latent bugs. The Java implementations do impact performance, although I believe this has improved with the module system, which uses a URN instead of a URL for codesources, I haven't looked at the jrt URL handler impl. Regards, Peter. On 31/05/2021 4:21 pm, Harshad RJ wrote: In reply to Ron Pressier: Do you regularly use the Security Manager to sandbox your own dependencies and find it convenient and effective — in which case, could you please describe your practice concretely so that it would be possible to consider alternatives — or are you saying that you can *envision* such a powerful use-case? The desire to remove the Security Manager does not stem from its theoretical utility, which is absolutely amazing, but from its practical utility, which years of experience have found to be less than amazing after all, and probably too low to justify its burdensome cost. # Practical example of SecurityManager use We are developing a security and privacy conscious browser in pure Java: https://gngr.info `gngr` uses `SecurityManager` at its core to not only sandbox external libraries but also *internal modules*! # External Libraries we use * `Rhino` for Javascript * `jStyleParser` for CSS parsing and analysing * `okhttp` for HTTP/2 support (before the Java11 HTTP client was available) These are huge libraries that are practically impossible to audit by a small team, but through the SecurityManager we were able to identify the following issues very easily: ## Security issues discovered so far * Network access by the CSS parser library: https://github.com/radkovo/jStyleParser/issues/14 * Use of reflection to make a private method accessible https://github.com/square/okhttp/issues/3426 # Trust, but verify We don't use `SecurityManager` because we don't trust the external library authors; we do trust them else we wouldn't be using those libraries! And given that these are popular projects, we can also bank on oversight from the community at large. But what if we could also verify this trust and other assumptions, for very little overhead? The SecurityManager does require initial time investment, but IMO the returns justify it. A good example of trust but verify, is our own internal modules. We obviously trust our own code, yet we sandbox it, to be sure that our assumptions are correct. For example, the `Cookie` m
Re: JEP: 411 How JGDMS utilises the Java platform
Something else that JERI does that RMI doesn't is transfer the Subject from a calling thread in one JVM to the thread invoked in another JVM, so you have user principles, and service principles mixing in numerous JVM, all while only granting the minimal permissions required to perform role / task. On 30/05/2021 9:22 am, Peter Firmstone wrote: This is provided as information only I'm not looking for arguments, debate or even requesting functionality, I'm just documenting what JGDMS does now, unfortunately JGDMS appears to be too entangled with current security API's at its foundations to be easily migrated, I guess that will depend on the new API's, but it might provide some insights into the next security architecture's design. My fear is this JEP is focused on removal, more than replacement. There is no limit on the number of nodes that may participate, nor the number of principals. JGDMS is a service based architecture, these services are discovered dynamically, I'm not going to go into the details of discovery here, but instead to focus on the architecture of the service and its components. Firstly while the service itself is a "Server - Client" model, the JVM's or compute nodes are peer to peer and doesn't directly translate to a distinction between server and client compute nodes, Swing is supported for graphical API's, it would also be possible to use JavaFX also (untested). When a node wants to participate, it discovers lookup services, that belong to particular groups, these lookup services contain services available, the node may register services with these lookup services, and it may search for other services and utilise them. So back to focusing on the Service: Below is a diagram of a service in a modular environment, where Bundle is an OSGi module, but it could also be a JPMS module (future version) for arguments sake. Here the COM link is a remote invocation protocol, but may be any of a number of protocols, the protocol can be replaced without modification to the service implementation via configuration. It is possible make use of, but doesn't require RMI or Java Serialization, any number of protocols may be used. These are Remote Invocation protocols. The most commonly used communication protocol is called JERI, or Jini Extensible Remote Invocation, JGDMS provides a JERI implementation with an Atomic Invocation Layer, where it supports failure atomicity. Unlike RMI, Atomic JERI assigns ClassLoader's at each endpoint for class resolution and it doesn't append CodeSource URL's into the stream. Before the Client JVM can load the Proxy Bundle, it must first designate a ClassLoader, this ClassLoader has an identity, which is made up of, RFC3986 URI CodeSource's, The Servers Endpoint identity (which includes the servers IPv6 address and the security constraints made) and the type of connection used (eg TLS, or Kerberos). Permissions are granted statically using POLP, statically by policy files (or remote policy services) to each ProtectionDomain, based on CodeSource and Principals, and dynamically to the ClassLoader of the ProxyBundle in the Client JVM, following server and client authentication. Different services may even use the same CodeSource, however their classes are loaded by different ClassLoader's based on the Server's identity and security constraints. So while OpenJDK devs claim the ProtectionDomains on the stack represent Code and Principles, we use it to represent the difference services on the call stack, not code. In our case spawning hundreds of processes to manage the permissions of each service used by a client isn't practical. I hope this helps people understand how we are using Java, this is only scratching the service, we are not the only ones who do this, I am aware of companies and military, who use POLP also. |SERVER JVM __ | | | Service | | API Bundle | |__| | | Imports API Packages | | __|___ | | Imports packages | | | Service Bundle |<| Proxy Bundle |--SEP | Implements | from proxy |__| | | Proxy API | | || | | | | | | C O CLIENT JVM M | L __ I | | | | N | Distribution | Imports packages | Service | K | Provider |<| API Bundle | | | Discovers & | from API |__| | | Registers service | | | || | | Imports | API | Packages | | | | | __|___ | | | | | Proxy Bundle |---EP |__| | -- Regards, Peter Firmstone
JEP: 411 How JGDMS utilises the Java platform
This is provided as information only I'm not looking for arguments, debate or even requesting functionality, I'm just documenting what JGDMS does now, unfortunately JGDMS appears to be too entangled with current security API's at its foundations to be easily migrated, I guess that will depend on the new API's, but it might provide some insights into the next security architecture's design. My fear is this JEP is focused on removal, more than replacement. There is no limit on the number of nodes that may participate, nor the number of principals. JGDMS is a service based architecture, these services are discovered dynamically, I'm not going to go into the details of discovery here, but instead to focus on the architecture of the service and its components. Firstly while the service itself is a "Server - Client" model, the JVM's or compute nodes are peer to peer and doesn't directly translate to a distinction between server and client compute nodes, Swing is supported for graphical API's, it would also be possible to use JavaFX also (untested). When a node wants to participate, it discovers lookup services, that belong to particular groups, these lookup services contain services available, the node may register services with these lookup services, and it may search for other services and utilise them. So back to focusing on the Service: Below is a diagram of a service in a modular environment, where Bundle is an OSGi module, but it could also be a JPMS module (future version) for arguments sake. Here the COM link is a remote invocation protocol, but may be any of a number of protocols, the protocol can be replaced without modification to the service implementation via configuration. It is possible make use of, but doesn't require RMI or Java Serialization, any number of protocols may be used. These are Remote Invocation protocols. The most commonly used communication protocol is called JERI, or Jini Extensible Remote Invocation, JGDMS provides a JERI implementation with an Atomic Invocation Layer, where it supports failure atomicity. Unlike RMI, Atomic JERI assigns ClassLoader's at each endpoint for class resolution and it doesn't append CodeSource URL's into the stream. Before the Client JVM can load the Proxy Bundle, it must first designate a ClassLoader, this ClassLoader has an identity, which is made up of, RFC3986 URI CodeSource's, The Servers Endpoint identity (which includes the servers IPv6 address and the security constraints made) and the type of connection used (eg TLS, or Kerberos). Permissions are granted statically using POLP, statically by policy files (or remote policy services) to each ProtectionDomain, based on CodeSource and Principals, and dynamically to the ClassLoader of the ProxyBundle in the Client JVM, following server and client authentication. Different services may even use the same CodeSource, however their classes are loaded by different ClassLoader's based on the Server's identity and security constraints. So while OpenJDK devs claim the ProtectionDomains on the stack represent Code and Principles, we use it to represent the difference services on the call stack, not code. In our case spawning hundreds of processes to manage the permissions of each service used by a client isn't practical. I hope this helps people understand how we are using Java, this is only scratching the service, we are not the only ones who do this, I am aware of companies and military, who use POLP also. |SERVER JVM __ | | | Service | | API Bundle | |__| | | Imports API Packages | | __|___ | | Imports packages | | | Service Bundle |<| Proxy Bundle |--SEP | Implements | from proxy |__| | | Proxy API | | || | | | | | | C O CLIENT JVM M | L __ I | | | | N | Distribution | Imports packages | Service | K | Provider |<| API Bundle | | | Discovers & | from API |__| | | Registers service | | | || | | Imports | API | Packages | | | | | __|___ | | | | | Proxy Bundle |---EP |__| | -- Regards, Peter Firmstone
Re: JEP 411: Missing use-case: user functions in an RDBMS
While I accept that my particular use case will no longer be supported in future, it's difficult to see the value of a sandbox, because developers will always want to relax it in some way and people fall into the trap of thinking that trust is black and white; this is trusted, that is not. Nowadays, there's a lot more interest in the Principle of Least Privilege, especially in health care regulations, now I'm going to use the meaning implied in "Inside Java 2 Platform Security, Second Edition", so that we aren't arguing whether that applies with the JVM or external to it. The reason POLP is simpler, is that it can be automated with tooling, then the development / deployment team may choose to relax it, they can certainly test it and validate it. Then we are simply focused on what we need to do, rather than what we might want to restrict, which is always unknown and subject to change. The problem is that this is not how developers have been taught to use SecurityManager, instead they are told that trusted code can be given AllPermission because it's trusted and that's an approach which has proven largely ineffective and the OpenJDK development team carry's the cost as a result, with little benefit. I just thought I'd provide you with a different perspective, so hopefully the mistake isn't repeated. Peter. On 28/05/2021 8:09 pm, Ron Pressler wrote: Deep sandboxes, simple or stack-dependent, are useful for very “rich” code, that is potentially very big and possibly contains arbitrary third-party libraries, while shallow sandboxes are more suitable to limited plugins. While a complex, stack-dependent, deep sandbox *could* be used for plugins, permissions that don’t specify what is forbidden but what is allowed effectively also severely limit the use of third-party libraries, that for example, might want to do benign operations with their own files, and so effectively only allow very limited plugins. The result is a costly mechanism that is overkill for what it’s used for. — Ron
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
nces of finding fault with it are low. It wasn’t put up after someone just had an idea. It is, therefore, not entirely surprising that no one has been able to give any relevant arguments against it. Clearly, you have had far less time to think about it, but I can’t seem to steer your arguments in a direction that’s relevant to what is actually the issue. I understand you’re invested in the Security Manager and that its removal would impose real costs on you. I am even willing to believe that you actually believe that, despite what studies show, despite what experts say, despite what the developers of Java say, despite what the developers of *all* other mainstream software platforms with an emphasis on security, both new and old —- from .NET to WebAssembly —- have concluded after decades of experience, that the Security Manager is not only the best way to secure Java but apparently the only one. Maybe you’re right and those others are wrong, but please accept that we all want to improve Java’s security, we just disagree with you on the best way to do it. Yes, additional security measures, whatever they are, would provide additional security. But if the choice is between measure A and measure B -— you *can’t* have both —- you pick the one that is *more* effective per cost. None of your arguments so much as glance in that direction, and they don’t acknowledge the fact that Java security would be hardly affected by the Security Manager’s removal even without better protection elsewhere for the simple reason that it is hardly ever installed, including on the most security-critical applications, whose defences apparently aren’t so feeble even today. In any event, if the question is, do we want a perimeter fence and security cameras *XOR* locks on all room doors, the argument that they provide security through different mechanisms so we should have both completely misunderstands the question. I am also confused by your point about multi-user applications. Of course different users have different access, but surely you are aware that very few applications do that using the Security Manager, which isn’t needed —- and is rarely used -— for that purpose. Short of making relevant arguments, I would urge you again to focus on suggestions to reduce the harm this proposal would cause you. — Ron On 22 May 2021, at 00:17, Peter Firmstone wrote: I had hoped by end of this discussion, that there would at least be an understanding of what OpenJDK is so hastily choosing to destroy. Once it is gone, it will be irretrievable, it will never be possible to lock down the JVM so securely again. On 21/05/2021 11:06 pm, Ron Pressler wrote: On 21 May 2021, at 12:52, Peter Firmstone wrote: It's quite clear this will be pushed through anyway, No, not *anyway*, but given the fact that the community consists of millions of users, this proposal has been well-publicised, I discovered the proposal on the 11th of the May on a mailing list I was subscribed to and I almost missed it. Yes, it will be pushed through regardless, clearly the decision was made before publication. Everyone saw applets coming, if the developers were serious about supporting applets, they would have designed a stripped down subset of Java, a JVM specifically suited that task which, didn't include things like XML or serialization. Just think, Applets were killed because of their atrocious security. How ironic. The granularity is not arbitrary, you said by class, which is incorrect. Granularity is by a combination of one or more of the following: • ProtectionDomain • CodeSource • Code signers • ClassLoader • Principals. What I said is correct. Assigning a ProtectionDomain to a class is possible, though not to a method (certainly not in code you can’t modify). In fact, ProtectionDomain is defined as “a set of classes,” i.e. class granularity. In particular, that is the granularity that instrumentation with doPrivileged aims to address, and that is one of the Security Manager’s most defining features. It may be possible to assign a ProtectionDomain, to a single class, but that doesn't make your assertions correct, you should be quoting common use cases, I have never seen an example of assigning permissions to a single class, besides, it requires a dynamic policy to do that, and Java doesn't have one by default, so you can't use PolicyFile to assign it permissions. Maybe you could use it to encapsulate ObjectInputStream with no permissions, then no one could grant it permissions, so that would be useful for Security. It doesn't change class resolution or visibility. But OpenJDK didn't do that, why not? What use case would there be to assign a ProtectionDomain to a method? Just use a permission check in the method, or wrap a sensitive class with a decorator before publication: https
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
I had hoped by end of this discussion, that there would at least be an understanding of what OpenJDK is so hastily choosing to destroy. Once it is gone, it will be irretrievable, it will never be possible to lock down the JVM so securely again. On 21/05/2021 11:06 pm, Ron Pressler wrote: On 21 May 2021, at 12:52, Peter Firmstone wrote: It's quite clear this will be pushed through anyway, No, not *anyway*, but given the fact that the community consists of millions of users, this proposal has been well-publicised, I discovered the proposal on the 11th of the May on a mailing list I was subscribed to and I almost missed it. Yes, it will be pushed through regardless, clearly the decision was made before publication. Everyone saw applets coming, if the developers were serious about supporting applets, they would have designed a stripped down subset of Java, a JVM specifically suited that task which, didn't include things like XML or serialization. Just think, Applets were killed because of their atrocious security. How ironic. The granularity is not arbitrary, you said by class, which is incorrect. Granularity is by a combination of one or more of the following: • ProtectionDomain • CodeSource • Code signers • ClassLoader • Principals. What I said is correct. Assigning a ProtectionDomain to a class is possible, though not to a method (certainly not in code you can’t modify). In fact, ProtectionDomain is defined as “a set of classes,” i.e. class granularity. In particular, that is the granularity that instrumentation with doPrivileged aims to address, and that is one of the Security Manager’s most defining features. It may be possible to assign a ProtectionDomain, to a single class, but that doesn't make your assertions correct, you should be quoting common use cases, I have never seen an example of assigning permissions to a single class, besides, it requires a dynamic policy to do that, and Java doesn't have one by default, so you can't use PolicyFile to assign it permissions. Maybe you could use it to encapsulate ObjectInputStream with no permissions, then no one could grant it permissions, so that would be useful for Security. It doesn't change class resolution or visibility. But OpenJDK didn't do that, why not? What use case would there be to assign a ProtectionDomain to a method? Just use a permission check in the method, or wrap a sensitive class with a decorator before publication: http://svn.apache.org/viewvc/river/permission_delegates/src/main/java/org/apache/river/api/delegates/package.html?view=markup But these are corner cases. More useful cases are for isolation, such as JEE. Restricting access by principal at the application level does not require the Security Manager, so that part is irrelevant, and, in fact, not only Principal, but also Permission, and even CodeSource and ProtectionDomain are *not* being proposed for terminal deprecation or even deprecation by this JEP. I guess your use case is a desktop application running in a single process? What about a multi user server application running in a single process? Now we have to spawn multiple processes for each user, that's hardly efficient or performant is it? I would like to understand this pain that is being caused to a far greater number of people? So far information has been scarce and it seems more of an excuse, as it's very light on detail. I would guess it's the pain of having to update policy files and making sure tests pass with security enabled. The pain is that the high cost of maintaining the Security Manager comes at the expense of other security measures that, we believe, provide far more security value to the Java ecosystem as a whole. Such as? I think the results of locking down the JVM to principles of least privilege are totally worth it and a saleable commodity in the current global environment. I absolutely accept the principle of least privilege. I do not accept that the marginal cost/benefit of applying it at class granularity yields its best application. I agree that there's little value for class granularity, but you are applying a corner case that although possible, is never applied in practice, and applying it with a broad brush, then using it as an argument against, please stop making this false assertion. Just because you can do something, doesn't mean that you should. Just because you can walk in front of a passing train, doesn't mean you should sir. There is however a significant benefit for applying the principle of least privilege. It can be assigned to Principal and Code signer granularity, that's actually quite coarse grained. It's very flexible, unlike white listing Serializable classes. Sure, theoretical things might, but there's no implementation in exi
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 21/05/2021 9:14 pm, Ron Pressler wrote: On 21 May 2021, at 11:29, Peter Firmstone wrote: Can you share this list please? If I see anything missing I can add it for you. No, because this might give the false impression that this is a debate. It's quite clear this will be pushed through anyway, so there's no harm in sharing the information, unless you don't have a list, in which case it's not possible to share something you don't have. So I'm restricting permissions granted to only those required to perform the intended tasks of the software, restricted even to a subset of which that software is capable, and you say it is not the principle of least privilege? That is not what the Security Manager is doing, though. It is able to assign “least privilege” to some arbitrary level of granularity and not others. The granularity is not arbitrary, you said by class, which is incorrect. Granularity is by a combination of one or more of the following: 1. ProtectionDomain 2. CodeSource 3. Code signers 4. ClassLoader 5. Principals. How do you critique something if you don't understand how it works? I stand by what I said. “X granularity” means up-to and no-less-than X, and assigning software privileges per user at the application level does not require the Security Manager. I understand that your particular codebase does use the Security Manager to assign all software privileges, and that removing it might require costly changes to your codebase, but that is not the most common way, and it is certainly not the *only* way this is done in Java codebases. I absolutely sympathise with the pain this proposal would cause you, but please sympathise with the pain that not accepting it would cause what we believe is a far greater number of people, and don’t try to universalise your particular situation. You said class level granularity, which is incorrect. True, it is not possible to upgrade, I guess you could say that's painful, as it will be for everyone else caught up by it. I would like to understand this pain that is being caused to a far greater number of people? So far information has been scarce and it seems more of an excuse, as it's very light on detail. I would guess it's the pain of having to update policy files and making sure tests pass with security enabled. I perform all tests with security enabled, because it will work if it isn't anyway. Policy files are easily regenerated using tools. What other pain is there? I think the results of locking down the JVM to principles of least privilege are totally worth it and a saleable commodity in the current global environment. So you are mistaken about what it is that I am critiquing. I am not critiquing the operation of the Security Manager nor am I saying that it is incapable of preventing certain attacks. I am critiquing the claim that no other technique can achieve better security more cheaply. Sure, theoretical things might, but there's no implementation in existence. It has been quite affordable for me, so I wish to understand this pain, because I currently don't, I'm already using the latest encryption, static analysis, secure coding practices, validating input, sanitizing data etc. I am saying that the marginal utility if the Security Manager divided by its marginal cost is lower than the marginal utility divided by the marginal cost of other security techniques. Other techniques that are yet to be developed. OpenJDK is deprecating SecurityManager prior to the implementation of it's replacement, a little more notice would have been nice. I'm ready for you to deprecate Serialization, we saw that coming, but this is just completely unexpected out of left field. We have no argument over the value of security or that of the principle of least privilege. We merely disagree over whether the Security Manager specifically is the best possible means to those ends. Currently it is the only possible and practicable means that has been implemented, I'm sure something better will come along, but everything better is just theoretical at this time. Nothing else is even remotely capable of locking the JVM down this securely. I don't wish to publicly show that I doubt your credibility, but you are making it difficult for me not to sir. It is your user base you need to convince, so far, you are not very convincing. I am sorry, but that not everyone would be convinced by every argument we make is something we take as a given. It is the duty of those who disagree with proposals put forth by the maintainers to convince the maintainers, not the other way around. I am trying to help you focus you on arguments that would be relevant. They are: 1. The *current* value the Security Manager to the Java ecosystem is high, evidenced by the great number of companies t
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 21/05/2021 6:58 pm, Ron Pressler wrote: These are just opinions, it's nice to have them, but they're not helping. I think you'll find usages of SecurityManager api's are far more widespread than your opinion suggests, but that's just my opinion too lol. I guess you could say they’re my opinions because there is no conclusive proof, but they are based on empirical data. At least you should scan GitHub and Maven Central. That few libraries are properly instrumented is yet another piece of evidence. This isn’t conclusive, and that is why we’re discussing this here in an attempt to collect more information, but we have done our homework. Can you share this list please? If I see anything missing I can add it for you. Yes, I think we should keep it simple, and keep calling it the principle of least privilege, to avoid confusion, it's very beneficial for reducing the consequences of perimeter security failure, even if your experts think otherwise. It is absolutely not the principle of least privilege, So I'm restricting permissions granted to only those required to perform the intended tasks of the software, restricted even to a subset of which that software is capable, and you say it is not the principle of least privilege? Please refer this policy file: https://www.dropbox.com/s/0w1c140zts93cmw/defaultnonactvm-policy.txt?dl=0 It's even specific to the JVM used, vendor, version and installation path. Notice how the attached grants permission to code and principal? That means the code without the Principal cannot attain that privilege, nor can the Principal use that permission without the specified code? This means if an attacker breaches peripheral protection systems, such as authentication, they can not perform anything that requires permission outside of the granted scope? I execute exactly the code paths I want executed, with the user role I want to perform that task, I grant only the permissions required to perform the intended task. How is this absolutely not the principle of least privilege? Java (the last remaining mainstream platform with a mechanism that assigns permissions on a per-class basis). Java Policy doesn't assign permissions on a per-class basis, it assigns them by ProtectionDomain, or by ClassLoader, or by CodeSource, or by Signer Certificates, or by Principal, or by a combination of Principal's and (ProtectionDomain or ClassLoader or CodeSource, or signer Certificates) and Certificates. I don't really want to document all the possibilities here if yo don't mind. How do you critique something if you don't understand how it works? With test cases, I can generate policy files for the principle of least privilege for any Java software, including dependencies. Not only that, but I do it without impacting performance or scalability. Show me some evidence for your arguments, name your experts you speak of so often. If you are going to argue, back it up with hard evidence, not nonsense. I provide you with hard evidence, show me the same courtesy and respect please. I'm not seeing any evidence that you have done much homework at all sir? Previously you claimed SecurityManager didn't even work, that a Doctorate in Nuclear particle physics was required to use it. I don't wish to publicly show that I doubt your credibility, but you are making it difficult for me not to sir. It is your user base you need to convince, so far, you are not very convincing. -- Respectfully, Peter Firmstone
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 18/05/2021 10:09 pm, Ron Pressler wrote: On 18 May 2021, at 12:27, Peter Firmstone wrote: However I disagree that the Principle of least privilege is wrong headed, I think you've been discussing sandbox concepts with the experts and they're going to tell you that's a bad idea. But the two aren't the same, one is access control, the other is attempted complete leak proof isolation, which is a very difficult thing to do that grows exponentially with your API surface area, I'm sure in years to come crackers will find vulnerabilities in VM's too. Sandboxing would work if you removed most of the library code and features and limited memory and threads, but I don't think anyone is seriously considering that. Java for applets should have been it's own stripped down limited JVM. When you say the “principle of least privilege” you seem to imply that it is at *all* cost. Many believe that the attempt to assign different privileges to different *pieces of code* (not different people) is not effective from a cost/benefit perspective. In other words, the effort would have a bigger impact on security if directed elsewhere. After all, the same principle would imply that we’d need to afford different permissions at a method granularity rather than class granularity. No, actually to date, it's been a very affordable option, probably because the burden has been on Openjdk developers to date, to which I'm very grateful. It doesn't impact performance and it allows me to reduce the attack surface of my programs, especially in regard to dependencies. https://debricked.com/blog/2021/01/02/vulnerabilities-in-dependencies/ What sort of cost, you mean development cost, the cost of understanding doPrivileged? Both, but mostly the former. Yes everything is a compromise and there are trade-offs. The way I see it, the cheapest way to maintain security is to find others who share a common pain point is to maintain a copy of OpenJDK focused on security. We can audit upstream changes and add them later, with the security cost penalty, without throwing out the huge amount of effort performed by JDK developers over the years, there's no way we can re-implement it economically, so this appears the most sensible option. At least to me, it's just a simple lambda expression, it's not difficult. It think people are making this more complex than it needs to be due to a proliferation of Permissions, but even that I manage with tooling. There’s no point to argue over this. It’s been tried for a couple of decades, people don’t want it, experts say it’s not cost-effective, no popular platform after Java and .NET has chosen to invest in it, and .NET has dropped it already. Even if you convinced me that the effort of attempting to afford different permissions to different code in the same process was an effective use of resources, if you could have changed the habits of millions of developers you would have done it by now. You can carry on the argument over whether this is a good or bad idea on security forums and at security conferences, but that it didn’t pan out, adoption-wise, in practice is an empirical fact (that it's worked great *for you* is completely irrelevant). Even good ideas can fail to gain traction, and we shouldn’t throw good money after bad. These are just opinions, it's nice to have them, but they're not helping. I think you'll find usages of SecurityManager api's are far more widespread than your opinion suggests, but that's just my opinion too lol. You've probably have never used it, so you don't really understand it, I can't argue with ignorance and you can't understand the benefit, if you are only asking others for their opinions, you are likely surrounded by people with the same opinion, so you're suffering from confirmation bias without any practical application, as is anyone else that doesn't use it. https://berteig.com/coaching/confirmation-bias/ But it's such a shame to lose access control as collateral damage. Most applications that do access control today do not use the Security Manager for that. When you say “access control” you mean something very specific, which is not what others mean when they say access control. In particular, even doAs is proposed to remain. — Ron Yes, I think we should keep it simple, and keep calling it the principle of least privilege, to avoid confusion, it's very beneficial for reducing the consequences of perimeter security failure, even if your experts think otherwise. -- Regards, Peter Firmstone
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
If there are those of us who wanted to maintain a fork of Java 17,
focused on security, we could backport new features after they've been
reviewed for security.
Would we be welcomed to do that here? Otherwise is it something we
should do on GitHub?
Cheers,
Peter.
On 21/05/2021 11:25 am, David Black wrote:
On Thu, 20 May 2021 at 21:27, Andrew Dinn wrote:
On 18/05/2021 23:06, David Black wrote:
I don't think that this thinking is unique but it might not be worth
the "cost" to Oracle to maintain something that seemingly for various
reasons Oracle has little interest in maintaining (we're not in
applet-land anymore). I would like to encourage proposals that mean
that people who want to do 4, who implement further security hardening
where others seemingly shy away, can continue to do 4.
Please don't do that. The cost Ron is talking about is not to "Oracle".
It is a cost to the OpenJDK project as a whole.
Sorry about that, that is a good point.
Likewise, the lack of project team interest in maintaining the security
manager and self-evident interest in applying resources to providing
other, more valuable Java capabilities is not simply restricted to
"Oracle" project members.
regards,
Andrew Dinn
---
Red Hat Distinguished Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill
--
Regards,
Peter Firmstone
0498 286 363
Zeus Project Services Pty Ltd.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 18/05/2021 10:21 am, Ron Pressler wrote: On 18 May 2021, at 01:11, Peter Firmstone wrote: Your ideas are great in theory, in practice, the problem with your Agent proposal is every JVM release needs to be reviewed, and we have to review Java's internal implementation code, and understand it in order to instrument it. Absolutely. But that is exactly the work OpenJDK maintainers are required to do today to support something most people want better alternatives for at the expense of those better alternatives and other work. The Principle of Least Privilege reduces the consequences resulting from attacks that penetrate external security defenses, whether by social engineering, failure to sanitize input, protocols, or platform vulnerabilities. After many years of pain, JDK development work has not been wasted, I for one am grateful for the development efforts and thoughts put into securing the JVM. It is expensive and now your employer has decided they no longer wish to carry this expense, pushing it downstream. Java has a proven and well tested security API, it's flaws are generally well understood. At sometime in the future, Java's internal security defenses will be removed, and new code will no longer perform permission checks, so any breaches of external defenses will result in a JVM being Pwned. After considering the proposals for new Security API, I have decided that the Java 1.8 to Java 1.17 versions will be the most suitable as it has the best well developed and understood security architecture, requiring little further development work. The new proposals arising from this JEP present a significant development upgrade cost, and these new API's that we need to use for building new security architecture around, won't be battle hardened, are experimental and haven't passed the test of time. My assessment is the cost to upgrade is too high and far too great a risk in this instance, the least cost and safest option is to stay with well proven, deployment tested, high performance, hardened existing API's. I wish the OpenJDK project success in their future endeavors. We will try to support later Java versions in Service implementations, similar to other languages, however these will not be able to consume other services. This may enable users to take advantage of later language features. However we will always require the use of Java 1.8 to 1.17 as service consumers. I am grateful for the efforts, which appear to be thought of as wasted efforts or bad money, however this time spent addressing security issues, will leave the series of Java versions from 1.8 to 1.17 as some of the most secure versions of any software platform, as an asset of significant value to those who value security. Thank you. Peter Firmstone.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 18/05/2021 8:49 pm, Ron Pressler wrote: On 18 May 2021, at 03:39, Peter Firmstone wrote: Is it also possible to consider directing file access and network access through single points of access? This will simplify the process so we don't need to scour the entire codebase for usages. Of all your suggestions, I think this is the one that will be seriously considered from a cost/benefit perspective, although probably not as part of this JEP. Thank you ,appreciated. What about doPrivileged calls? Will they remain in existing Java library code, so we can utilise them? To avoid viral permission propagation? Doubtful. That is where much of the cost is, and it would mean investing significant resources into supporting a use-case that what seems like the vast majority of security experts think is a wrong-headed approach, for the sake of the few who disagree. Even as an additional mechanism that might, maybe, block exploiting some vulnerability as a result of some particular bug in some cases, sometimes, maybe, the high cost doesn’t justify what we believe is the extra defence gained compared to the gain of such an effort directed elsewhere. — Ron Oh well, there was no harm in asking. I agree that sandboxing is the wrong approach, it's unfortunate that this wasn't seen as a potential issue in the early days. It's similar with Java Serialization, if they tried to do less initially it would have been much better, but they pushed to envelope too far. But I'm pretty sure we've moved on from discussing sandboxes, no one is claiming that as their use case. However I disagree that the Principle of least privilege is wrong headed, I think you've been discussing sandbox concepts with the experts and they're going to tell you that's a bad idea. But the two aren't the same, one is access control, the other is attempted complete leak proof isolation, which is a very difficult thing to do that grows exponentially with your API surface area, I'm sure in years to come crackers will find vulnerabilities in VM's too. Sandboxing would work if you removed most of the library code and features and limited memory and threads, but I don't think anyone is seriously considering that. Java for applets should have been it's own stripped down limited JVM. Our problem is we will need to grant remote third party's AllPermission because they are not represented as a Subject based ProtectionDomain in the JVM, but then we could implement some sort of analysis that removes their ability to connect at all if they violate policy. But this doesn't seem very workable. Thankfully we have a very good, functional and highly scalable solution available now. What sort of cost, you mean development cost, the cost of understanding doPrivileged? At least to me, it's just a simple lambda expression, it's not difficult. It think people are making this more complex than it needs to be due to a proliferation of Permissions, but even that I manage with tooling. AccessController.doPrivileged uses 1.3% of CPU time in my system, and I'm sure I use it far more than many, so I don't think your talking about performance. Thankfully access control is much simpler today than say programming concurrently, but I try to do both well. I truly don't find it difficult to use, but then I've built a bunch of tools that make it useable. But I get that JVM developers have suffered a lot of pain with Security vulnerabilities due to Java Serialization and the Sandbox and just the mention of SecurityManager is received negatively. But it's such a shame to lose access control as collateral damage. Thanks again for spending the time to discuss. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
Because people have been treating it like a sandbox. Since it will take a number of years, can we at least consider my proposal and give it a try? It may reduce the burden in the interim. So this step deprecates it for removal, can we create a JEP for replacing the SecurityManager with AccessAssistant while retaining AccessController and AccessControlContext? It's also true that Permisssion's are more numerous than necessary, because it has been treated like a sandbox. If we rename and treat it like access control instead, we can reduce the burden, simplify permissions, provide tooling and make it much better than it is today. We've been doing exactly that for years, and we knew the Java PolicyFile code was bad, but we never realised that JDK developers saw it in this light. If I had realised that , I would have stepped in years ago to fix it. Maybe we got complacent because nothing was ever removed from Java for a very long time. We've been using it silently and efficiently for years. Regards, Peter. On 18/05/2021 7:13 pm, Alan Bateman wrote: On 18/05/2021 08:36, Peter Firmstone wrote: : It's a perception issue, I understand, but we can fix that far less painfully. With respect, I don't see a viable route here. SM/AccessController and most of that security architecture has been an albatross around our necks for years. This JEP is the first step in pulling the JDK out of the sandboxing and policy enforcement business. It will take several releases and years to get there. Yes, it will be painful for those that have embraced this architecture but there will be years of supported releases to plan or develop alternatives. There is a wider group that have been using the SM as a means to intercept network, file and several other operations. This is an area that might need to be exploded further to see if an alternative solution is imported for the JDK to provider. We don't think that this needs to be fully explored and decided on before making progress on the deprecation. -Alan -- Regards, Peter Firmstone 0498 286 363 Zeus Project Services Pty Ltd.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 18/05/2021 4:10 pm, Alan Bateman wrote: On 18/05/2021 03:39, Peter Firmstone wrote: : Yes, I realize that, it is my understanding that because this is a security concern, it is not something the community is allowed to provide support for at OpenJDK, hence my suggestion to Alan, to make it possible for this to happen by changing the security level and calling it an access control layer concern. Sorry, I was too busy and didn't have time to reply to your previous mail on this and the performance anomaly in Java 14. I'm dubious about your suggestion. Every issue or report that Java has a security bug has to be analyzed on the assumption that it may be a vulnerability. Saying that SM bypass or a ninja move that disables the SM is not a security concern would create a perception issue. It would get lost in the discussion that the SM is rarely used in the way that it was envisaged 20+ years ago. The other thing is that it wouldn't remove the ongoing burden to insert/audit permission checks, check that ACC are captured and used in the right places, and of course testing as it's another execution mode. -Alan. Thanks Alan, It's a perception issue, I understand, but we can fix that far less painfully. You could create a new parent class of SecurityManager that's more aptly named, for example AccessAssistant, and deprecate SecurityManager for removal. That's actually pretty easy, even backport the new class to earlier versions of Java, so libraries can eliminate usages of SecurityManager sooner. This would give us time to change our dependencies to the superclass, and it eliminates the name perception issue. "AccessAssistant, is only intended to assist with access control measures to implement the principle of least privilege, while it may reduce the impact of a security breach, it's not capable of preventing it, therefore bugs in access decisions aren't given the same attention as security bugs, as both users and code are trusted." AccessController and AccessControlContext have functionality that we depend on for access control and to obtain user credentials for authentication. Let the community manage it as an access control layer, when we find the ACC has forgotten to be captured, we can fix it. When someone forgets to capture the ACC, the thread cannot establish a secure connection, it will get noticed and fixed, it alone cannot cause a security breach. We use TLS connections, sanitize data, atomic failure, input validation and authentication for security, we only use access controls and Policy for limiting access of trusted users and code and to obtain user credentials. Access control and POLP gets those boxes ticked and might be the difference for choosing Java over another platform. If people aren't defensively parsing their input, not using secure connections and authentication and someone takes advantage of that, well it's just too bad, it's their fault. If someone is using trusted connections but there's a successful phishing attack on user credentials, then the attacker can usually only obtain what the user has access to, yes it is possible to perform a privilege escalation attack, but the point is the breach has already occurred, the security vulnerability is not access controls, it was the authentication system failure. Don't stress the performance anomaly, it hasn't regressed in later versions and the standard Java Policy provider will mask it. You'll only notice it when using a scalable policy provider. If I see it again in any long term releases, I'll let you know. I suspect whatever caused the problem was spotted and fixed already. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 18/05/2021 10:21 am, Ron Pressler wrote: On 18 May 2021, at 01:11, Peter Firmstone wrote: Your ideas are great in theory, in practice, the problem with your Agent proposal is every JVM release needs to be reviewed, and we have to review Java's internal implementation code, and understand it in order to instrument it. Absolutely. But that is exactly the work OpenJDK maintainers are required to do today to support something most people want better alternatives for at the expense of those better alternatives and other work. Yes, I realize that, it is my understanding that because this is a security concern, it is not something the community is allowed to provide support for at OpenJDK, hence my suggestion to Alan, to make it possible for this to happen by changing the security level and calling it an access control layer concern. Also is OpenJDK using static analysis to spot bugs? Just out of curiosity, it's something we do for our code. POLP isn't perfect, I acknowledge that, but it's much better than nothing as it does assist to limit damage in the event of an attack. If this is the only issue, I'm sure the community will be willing to assist. It would be less work for me to maintain the existing implementation than try to re-implement this functionality in a relatively compatible way. I would review OpenJDK's permissions as well, given that I use doPrivileged and POLP regularly, it's probably going to be more obvious to me where bugs are, I do see evidence of these bugs in my policy file generation, so I would be able to fix a number relatively quickly. Unfortunately for us to put these proxy's into separate processes, in order to use OS level access control, we could have hundreds to thousands of processes running. So it's not really an option. We would still need to communicate with that process, so it's starting to make it difficult to manage secure connections etc. ClassLoader's are so lightweight in comparison to a process. Our RFC 3986 and RFC 5952 URI based ClassLoader is also so much faster than URLClassLoader, so it's not just security performance, I looked for hotspots and eliminated all of them. Where OpenJDK is doing dns network calls, we normalize ASCII strings. It's AL2.0 licensed code refactored from Harmony, so I don't think it's an option for OpenJDK. We'd like to avoid performance penalties as well, other options proposed have unknown performance costs, which may be significantly worse than current performant and scalable code. Remember our performance limitations are JVM native sockets, if you look at our hotspots, they're all JVM native methods. While I admit that what we do with distributed software isn't commonly done in modern software, as most gave it up as too difficult years ago, we persisted and solved some very difficult problems. Maybe if you put hooks (annotations?) into the JVM code, so it was easier for agents to know which calls need to be controlled for access decisions? But then if not many people are using it, it will suffer neglect. Yeah, it sounds neither here nor there, but the relevant maintainers will consider it. It may be the deciding factor for us, so I'd appreciate it if this can be considered. Is it also possible to consider directing file access and network access through single points of access? This will simplify the process so we don't need to scour the entire codebase for usages. What about doPrivileged calls? Will they remain in existing Java library code, so we can utilise them? To avoid viral permission propagation? It's your existing userbase with over 50% still using Java 8 that need convincing, who will be ultimate judge of the success or failure of this decision. If you have data that contradicts our estimate of Security Manager usage among Java 8 users, please present it. - Ron I don't know what data you have, it's not something I've considered polling, if I find anything, I will most certainly share it. -- Regards, Peter Firmstone
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
Your ideas are great in theory, in practice, the problem with your Agent proposal is every JVM release needs to be reviewed, and we have to review Java's internal implementation code, and understand it in order to instrument it. But I do appreciate you took the time to do your homework to make the suggestion. Maybe if you put hooks (annotations?) into the JVM code, so it was easier for agents to know which calls need to be controlled for access decisions? But then if not many people are using it, it will suffer neglect. To answer your use case question below, we use a ClassLoader and ProtectionDomain, to represent the remote server Subject, this is not for authentication, just for access control. JAAS is used for authentication of TLS endpoints. For services, the calling Subject is run in the context of the thread on the remote machine for access decisions also. It's your existing userbase with over 50% still using Java 8 that need convincing, who will be ultimate judge of the success or failure of this decision. Thanks for your time. -- Regards, Peter Firmstone On 18/05/2021 8:16 am, Ron Pressler wrote: On 17 May 2021, at 21:46, Peter Firmstone wrote: Yes, you are talking about those who maintain and develop OpenJDK, but this is only a small proportion of the overall Java developer ecosystem. Not at all. I’m talking about the millions of developers who don’t get what they need sooner because the OpenJDK’s resources are invested in maintaining components that add little value to the ecosystem. No, it's not practical, but there's no viable upgrade path, so our only option is indefinite maintenance of Java 17. I’m not going to tell you where to put your money, but writing a Java agent is orders of magnitude cheaper than maintaining an eight-million-line codebase indefinitely. In a distributed system that uses POJO's and Remote Method Invocation, a proxy is used to invoke methods on an object in a remote JVM or any other language for that matter, operating in another node or process. The proxy implementation is not known, nor necessarily are it's communication methods until runtime. It is loaded dynamically. The proxy is trusted code (each implementation has different bytecode), but we wish to run it with principles of least privilege, rather than an anything goes approach. https://urldefense.com/v3/__https://github.com/pfirmstone/JGDMS/wiki/OSGi-and-JGDMS__;!!GqivPVa7Brio!PAF0JUFBtHO0VgRqhSDtRI3p3J-zPmJvmwVDF7p7eXyLj3MoMOHjQfCKUEEqMlozJA$ I’m not sure I understand. If you’re not talking about a JDK mechanism, you can change that mechanism to not use the Security Manager, just as JAAS will be changed. If the application needs access control, but the Java platform doesn't provide it, and you must remove it for the application to run, then yes, the software will be less secure as a result. This shouldn't be difficult to understand. I understand but disagree. No, but it is a useful security layer to control access. The utility of *this particular mechanism* does not justify the cost of maintaining it. The question isn’t if such a mechanism could be useful or not, but is this the best use of resources to improve security? In other words, the choice isn’t between this and nothing, but between this and everything else. It is not possible to implement fine grained access control in Java, or as a library, this feature is low level JVM code. Again, when you say fine-grained level access control you don’t mean fine-grained level access control but the particular features provided by the Security Manager; and those features — at least those you sound most interested in — are not low-level JVM code, and can be implemented in Java with a Java agent, at least as an additional, albeit imperfect layer of security, if you think this is a worthwhile and cost-effective approach. Developers who wish to develop new applications that require access control should do so using another platform, those who have existing software deployments, need to stay on Java 17 for example, indefinitely until EOL. I strongly disagree; the Security Manager is not the only — and I would say it’s not even the best — way to offer access control. This JEP proposes removing Security Manager. That that implies giving up on access control is entirely your insistence on identifying the two. And if the only way you can think of implementing access control is with a call-stack code-sourcing mechanism, then I’m afraid other platforms are not going to offer it, because they also don’t share the view that this is the best, let alone the only, way to do this. New developments that require access control might consider the Microsoft CLR for example, however I'm no expert, there may be better alternatives. https://urldefense.com/v3/__https://www.c-sharpcorner.com/UploadFile/84c85b/net-code-access-security-cas
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 18/05/2021 12:25 am, Ron Pressler wrote: On 17 May 2021, at 13:47, Peter Firmstone wrote: It is a foundational feature, it has a significant impact on those who adopted it. True. But the problem is that it also has a significant impact on those who didn’t. Yes, you are talking about those who maintain and develop OpenJDK, but this is only a small proportion of the overall Java developer ecosystem. But I understand that this isn't insignificant either, just like the features removal isn't insignificant to those affected by it. Out of curiosity, was anyone here impacted by the oil pipeline shutting down? Impacts can be direct and indirect. This is an existing system, your arguments are not relevant as the cost of rewriting millions of lines of code is prohibitively expensive. I don’t think we’re talking about rewriting millions of lines of code, and the cost of keeping Security Manager is far greater, anyway. We want to shift the high cost of maintaining the SM, or something like it, from those who *don’t* want it to those who do, and I think that’s more than fair. No, it's not practical, but there's no viable upgrade path, so our only option is indefinite maintenance of Java 17. My point is that there will be no restriction on the services themselves in the JVM consuming and using the services. Currently service proxy's are only allowed to contact their originating server and negotiate a few required permissions for their operation. In future versions of Java without a SecurityManager, they will have no restrictions at all. I don’t understand this statement. Which JDK-implemented “service proxies” are you referring to that today implement such a restriction in the JDK? In a distributed system that uses POJO's and Remote Method Invocation, a proxy is used to invoke methods on an object in a remote JVM or any other language for that matter, operating in another node or process. The proxy implementation is not known, nor necessarily are it's communication methods until runtime. It is loaded dynamically. The proxy is trusted code (each implementation has different bytecode), but we wish to run it with principles of least privilege, rather than an anything goes approach. https://github.com/pfirmstone/JGDMS/wiki/OSGi-and-JGDMS Yes, it is the case for software that was designed to use the SecurityManager. We need to be honest about the impact, yes I understand SecurityManager will be removed, however telling developers their EXISTING software is no less secure is inaccurate. It is accurate, because if existing software picks a new runtime, then it will have to be modified in order to run at all, if only to acknowledge it is depending on a deprecated, and later, degraded feature. This JEP is proposing a responsible mechanism precisely to prevent situations like applications that do depend on SM for security suddenly become less secure without their knowledge. If the application needs access control, but the Java platform doesn't provide it, and you must remove it for the application to run, then yes, the software will be less secure as a result. This shouldn't be difficult to understand. So you could say, that the application has become less secure, with their knowledge, because it has a task to perform and it will be made so that it will continue to perform that task, even at the expense of less security, because security has become the obstacle to operation, which is the primary purpose, so security becomes a secondary concern. No one will switch it off if it's insecure, at least not until after they're hacked, then they'll look for bolt on security solutions and they may get approval to rewrite the system at that stage. Are you a security expert? Is this your opinion as a security expert? I am absolutely not a security expert, which is why I talk to security experts who tell me they do not recommend the Security Manager as a central security measure. No, but it is a useful security layer to control access. Security should be seen as a layered approach, rather than one solution fits everything. Each layer has it's role to play. The use of SecurityManager as a central security measure is not the intent. https://www.upguard.com/blog/principle-of-least-privilege Yes the Java Permission model is flawed, one cannot disallow an insecure network connection, while allowing a secure TLS authenticated connection, however we've created other ways of doing that, but it does rely on the Permission model to limit privilege. Your proposal is quite plain and simple, I don't see how it can be misunderstood, you propose to remove the ability to make stack based domain, access control decisions. Not the ability, but the built-in implementation. It is not possible to implement fine grained access control in Java, or
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
Some quick clarifications, I'll try to reply properly in the next few days. On 17/05/2021 9:29 pm, Ron Pressler wrote: On 17 May 2021, at 06:19, Peter Firmstone wrote: In versions of Java, without a security manager, the third party service provider will have AllPermission, and the user will have restricted permissions (if we still have some form of user Permission based access control). Follow this issue: https://bugs.openjdk.java.net/browse/JDK-8266592 Thanks for the link, I am aware of it. So basically we might as well remove all access control completely and say that all users and all code is completely trusted, All users — no, and at this point I’m starting to think that, rather than trying to understand the direction proposed here, which is ultimately meant to help make Java *more* secure, you’re trying to intentionally misunderstand and/or misrepresent it. I understand the direction and the reasons for the decision, however we still need to consider how it will impact those who utilise it, and I'm sure more examples will come to light in time. In my case, yes of course it will be less secure, because I'm using the stack context to manage access decisions for remote service identity. These service proxy's now have some very limited permissions, in future their access will be completely unrestricted. It is a foundational feature, it has a significant impact on those who adopted it. It does appear that a side effect of JEP 411, perhaps even an unintended consequence, will be to limit Java to trusted networks with one administrator. It is most certainly appears to be a single JVM focused change, or a system controlled by one administrator. Absolutely not. 99.99% of secure distributed systems in the world, written in Java or not, do not use Java’s Security Manager, and a great many of them mix of Java and other runtimes. You might have a point, though, that the current direction does not try to tailor a specific solution to distributed systems made *only* of JVMs, and, because systems are not controlled by one administrator, and because many are polyglot, mixing services running on different runtimes, this is very much the right direction to go. You, on the other hand, seem to be focused on “Java only” systems. This is an existing system, your arguments are not relevant as the cost of rewriting millions of lines of code is prohibitively expensive. Services written in other languages may be providers of services, however other language runtimes may not be consumers of services because most lack dynamic class loading or dynamic fine grained access control. The JVM is required to allow services to be agglomerated into a system. My point is that there will be no restriction on the services themselves in the JVM consuming and using the services. Currently service proxy's are only allowed to contact their originating server and negotiate a few required permissions for their operation. In future versions of Java without a SecurityManager, they will have no restrictions at all. Newer versions of Java will of course be less secure without access controls and unsuitable for use in a distributed system that involves more than one administrator. Of course not. Yes, it is the case for software that was designed to use the SecurityManager. We need to be honest about the impact, yes I understand SecurityManager will be removed, however telling developers their EXISTING software is no less secure is inaccurate. I realise you’re trying to paint a picture as if the removal of Security Manager, a barely used component, would adversely affects Java security — contrary to the opinion of security experts — b ut the fact is that the vast majority of Java systems today already use other security measures, including sandboxes. I don’t know if you actually believe this, in which case you misunderstand the proposal, or don’t believe it but think that such claims would sound convincing to others. It is true that we’re saying to those few remaining people who still depend on Java’s internal sandbox to do what most other people have already done and rely on other security measures, and so *if they do not* their systems will be less secure, but, of course, this is not what’s being recommended. All this JEP is saying that the JDK itself will not, in the long term, provide this particular fine-grained sandbox, and that remaining users should switch to other sandboxes available to Java applications. Barely used or not, it is only fair to acknowledge the impact on those who do use it. Are you a security expert? Is this your opinion as a security expert? I don't need a sandbox, I need access control. Your proposal is quite plain and simple, I don't see how it can be misunderstood, you propose to remove the ability to make stack based domain, access control decisions. These sandbox
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Although JGDMS is a different type of system, this is one of the functions of SecurityManager in our system also. We have methods equivalent to Subject.doAs, which instead of injecting the privileges of the user into all ProtectionDomain's, instead prepends a ProtectionDomain that represents the user's Subject onto the call stack. When used in this way, it changes policy file permission grant's, it actually simplifies them, because you can grant AllPermission to trusted code, if you want to, and then limit user permissions and it will restrict only the user, however this doesn't fully explain why we do this. https://pfirmstone.github.io/JGDMS/jgdms-platform/apidocs/net/jini/security/Security.html#doAs-javax.security.auth.Subject-java.security.PrivilegedAction- To explain this, requires briefly revisiting the 8 fallacy's of distributed computing, that is, item 6, there is only one administrator. So for your single JVM server, or many instances of JVM's operating independently, yes, there is only one administrator, but this is not the case for a distributed system. The reason we prepend a ProtectionDomain representing the user, is that there are ProtectionDomain's on the stack that represent Services other administrators are responsible for, it just happens to be proxy "code" that implements an interface, or interfaces, used for method calls between the two systems. These proxy's are assigned a ClassLoader based on the Server's security constraints, the server Endpoint address (by the proxy's InvocationHandler implementation) and the codebase annotation. So it's not really about the code, it's about the Service's identity and keeping the identities of different services separated from each other (even those using the same proxy code, with different server identity, eg for lookup services from different entities) and users, to avoid granting those Services all the users permissions while also allowing the user to authenticate with the service and also run as a logged in subject in threads on the remote node as well. As previously explained, the permission's required for Service proxy's are negotiated dynamically at runtime, between the two parties. We don't grant all the permissions the user has to the remote service provider, only those that have been negotiated, when a secure connection was first established, so the user allows the service provider to utilise some of the users permissions. The service provider cannot currently obtain more permission than the user has. In versions of Java, without a security manager, the third party service provider will have AllPermission, and the user will have restricted permissions (if we still have some form of user Permission based access control). So basically we might as well remove all access control completely and say that all users and all code is completely trusted, the system will be much more user friendly and easier to use, but legally that can present problems. It does appear that a side effect of JEP 411, perhaps even an unintended consequence, will be to limit Java to trusted networks with one administrator. It is most certainly appears to be a single JVM focused change, or a system controlled by one administrator. Newer versions of Java will of course be less secure without access controls and unsuitable for use in a distributed system that involves more than one administrator. In a way, it's ironic, IPv4 limited us to local networks, but now it looks like later versions of Java will now be the limiting factor. I realize less utilized platform features like access controls aren't the concern of OpenJDK developers, but it doesn't appear to be doing any harm talking about them, at least so the consequences of the decision can be better understood. I realize this is probably a business and marketing based decision. I guess Java has more of an enterprise history, and it's giving that up to become leaner and more developer friendly (less things to learn or understand). Cheers, Peter. On 17/05/2021 12:11 pm, David Black wrote: Hi Ron On Thu, 13 May 2021 at 20:22, Ron Pressler wrote: On 13 May 2021, at 03:11, David Black wrote: This seems somewhat more useful than 1 & 2 but imho it would be better to be able to perform checks/grant access on a call stack basis. This is an important point we’re trying to get across. The very reason the Security Manager was made this way is because it does *seem* better; certainly it is much more flexible. However, twenty five years of experience have shown us that *in practice* this is not the case, certainly not when you look at the ecosystem as a whole. It is hard to get right, which results in people not using the mechanism (which significantly reduces its utility and the value in maintaining it), or worse, use it and think it gets the job done, but actually use it incorrectly, providing the illusion of security without actua
JEP 411: Why the last version to support SecurityManager is likely to be the most powerful version of Java.
A disclaimer: I accept SecurityManager will be removed, but I feel a few words are in order before it's passing. A little history: Pre-1994, Bill Joy presented a proposal to Sun Labs where he presented three main concepts: 1) a language that would run on all platforms, 2) a virtual machine to run this language, and 3) a networked system to allow the distributed virtual machines to work as a singular system. In 1995, the language and virtual machine were introduced to the market as the Java Programming language and Java Virtual Machine. The system context, however, was kept in the Sun R&D for continued research and development. This system context is Jini. Reference: http://di002.edv.uniovi.es/~falvarez/whatisjini.pdf This fit into Sun's vision: "The network is the computer." The components of item 3 above, have presented significant technical challenges, progress having been made at various times during Java's history. Only in recent years have all the pieces of the puzzle come to together to realize the original goal. So it appears that Java 1.17 will remain an important version for many years to come. There are two types of scalability, vertical and horizontal. Various OS platforms that support different CPU and GPU architectures, clearly the vector API is an important component as are other recent improvements. IPv6 enables end to end connectivity, rather than IPv4's publish subscribe, server client model and issues with NAT. Web services are based on the publish subscribe model, with message passing. The Web services push API gives web applications the ability to listen to events. https://developer.mozilla.org/en-US/docs/Web/API/Push_API But there remain issues with cross site request forgery's. The glue is important too (which is what Jini is). What are the key components: 1. JVM's installed on each node. 2. Dynamic class loading, with verification and type safety. 3. Network end to end connectivity (IPv6 adoption is currently 32%) 4. Privacy, Authentication and Identity. 5. Object Serialization protocols (Java Serialization is flawed). 6. Jini Extensible Remote Invocation (RMI isn't suitable due to class resolution issues and limited extensibility). 7. Service architecture based on Interfaces. 8. Dynamic network discovery (IPv6 global discovery only recently added https://www.iana.org/assignments/ipv6-multicast-addresses/ipv6-multicast-addresses.xhtml). 9. Ability to "join" specific groups on the network. 10. Lookup services (multiple, it's a common misconception there is only one), to find services, as well as listeners to be notified when services are added, removed or changed. 11. Network Transaction Service. 12. A powerful Configuration, that uses Groovy. Why wasn't Jini a huge success when it was released in 1998? 1. It had some initial success, but class resolution was a big problem http://sorcersoft.org/resources/jini/smli_tr-2006-149.pdf as ClassLoader's added complexity that wasn't originally envisioned. To solve this problem, instead of using RMIClassLoading for class resolution and visibility, ClassLoader's are assigned at each Endpoint and are completely responsible for class visibility and resolution, so it works with non-hierarchical class loading systems like OSGi. 2. Security wasn't added until 2002, combined with IPv4, it relegated Jini to private trusted networks, in the age of the internet, limiting it's appeal. This is no longer the case. 3. When Security was added, it had been assumed the Java Sandbox and Java Serialization were secure, so authentication of services was performed after class loading and de-serialization, ouch! 4. The security model, added unnecessary complexity in the form of ProxyTrust (This isn't used today). We just authenticate first now, ensure the connection is private and validate the code, then let the code validate the input, to fail atomically, and this is handled for the user, all the user need do is configure their services to use TLS, and ensure their services runs with the login Subject. 5. Objects were exported remotely via their constructors allowing "this" to escape, prior to the Java 1.5 JMM, this was changed, an interface called Startable allows services to be started after construction, the same thing happens with Activation (Phoenix, not RMI), which is basically a watchdog for services. Sun later released Jini as open source under the Apache 2.0 license in 2007 and donated it to Apache who renamed it to River. Remaining issues have taken about 12 years to resolve, all up it's probably had 25 years of development. http://nighthacks.com/jag/res/Fallacies.html https://github.com/pfirmstone/JGDMS If JAAS continues to function in later versions of Java, we will be able to support them as server side services, but not as clients, without SecurityManager, so it will always depend on the last version of Java that contains a
Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 13/05/2021 10:59 pm, Sean Mullan wrote: The JEP does have a section on this: "In future JDK releases, we may degrade the Security Manager APIs so that they remain in place but have limited or no functionality. For example, we may revise AccessController::doPrivileged simply to run the given action, or revise System::getSecurityManager always to return null. This would allow libraries that support the Security Manager and were compiled against previous Java releases to continue to work without change or even recompilation. Once the compatibility risk has declined to an acceptable level, we expect to remove the APIs." So, if the JEP is targeted to 17, then the Security Manager will be deprecated for removal but will still be fully functional and supported in 17. *Disclaimer: The next part is forward thinking, and subject to change.* Once we start degrading the APIs, the functionality of the Security Manager may not fully work as before, so in that sense you might consider it "removed". We don't yet have a definitive timeline for that, it may occur in the next release, or it may not, but it will probably occur within a few releases after the release it is targeted to. --Sean It will be important for existing JAAS login mechanisms and AccessControlContext's for Subject's are still functional and preserved so that TLS clients and servers can authenticate. The AccessControlContext would only need to contain an array with only one ProtectionDomain, containing the Subject. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Thanks for confirming. Cheers, Peter. On 13/05/2021 10:59 pm, Sean Mullan wrote: On 5/13/21 6:00 AM, Ron Pressler wrote: On 13 May 2021, at 10:32, Peter Firmstone wrote: So it targets 17. I don’t know. I think that’s still TBD, but perhaps others know more. At this point, yes, we are planning to target the JEP to JDK 17. It would be nice to have certainty about which release it will be removed from, for planning purposes. Again it would seem that this isn't a consideration of OpenJDK. It very much is, which is why we have the deprecation and removal policy. Please read the JEP carefully. In addition to deprecation and removal, this JEP also proposes an interim step of degradation prior to removal. Removal, as the JEP says, will only be done once it no longer poses a big compatibility threat. At the fastest pace possible removal is more than a year away, though it will likely be longer than that. The JEP does have a section on this: "In future JDK releases, we may degrade the Security Manager APIs so that they remain in place but have limited or no functionality. For example, we may revise AccessController::doPrivileged simply to run the given action, or revise System::getSecurityManager always to return null. This would allow libraries that support the Security Manager and were compiled against previous Java releases to continue to work without change or even recompilation. Once the compatibility risk has declined to an acceptable level, we expect to remove the APIs." So, if the JEP is targeted to 17, then the Security Manager will be deprecated for removal but will still be fully functional and supported in 17. *Disclaimer: The next part is forward thinking, and subject to change.* Once we start degrading the APIs, the functionality of the Security Manager may not fully work as before, so in that sense you might consider it "removed". We don't yet have a definitive timeline for that, it may occur in the next release, or it may not, but it will probably occur within a few releases after the release it is targeted to. --Sean Is there an OpenJDK community project group that maintains older Java versions I can join? Yes, that would be the Updates Project. — Ron
Re: JEP411: Missing use-case: Monitoring / restricting libraries
I also think good developers will add these security features. The problem is that Permissions are currently used to regulate access control because this is how we've been told it should be done for over 20 years. Now maybe people aren't using the Java FilePolicy provider (for reasons stated previously), but as has been discovered, they often will override SecurityManager and implement some form of access control on trusted 3rd party code. Trusted code - Developed by people with good intentions - includes third party library code, we check it for known vulnerabilities at build time using OWASP, but we don't audit it ourselves as this requires understanding how the internals work and we are using it like a black box. If we have concerns we might perform static analysis, but we don't do that for dynamic code, eg maven dependencies and then we are powerless to do anything about it, other than report the bug, so access control can be used to mitigate until it's fixed, or we can use the principle of least privilege so we are not impacted by it. Java Serialization is also a form of parsing information, just like PDF documents and other file formats, problems with the Sandbox; there was no Permission to limit which classes could be deserialized, and JVM code always has AllPermission, so we couldn't restrict it either. Additionally there was no ProtectionDomain to represent the data being deserialized, and whether it was trusted or untrusted, these security features wouldn't have been difficult to add, however we rely on the platform developers to see the benefits. We are all only human. In JGDMS there are constraints we apply to services which require authentication and encryption, which must be met before deserialization or any downloads can proceed. In our re-implementation of Atomic Java Serialization, we use a DeserizablePermission, to limit which libraries can perform de-serialization. This is in addition to the use of failure atomic constructors, so we don't break object encapsulation. We also perform static analysis at build time and fix these bugs too. SecurityManager has much influence in access decisions in our software, we won't have the resources to upgrade, there's a massive amount of security auditing required before we could even consider it. Our only option is to participate in the maintenance of existing Java versions. Java will be our COBOL. We will be capable of using Java 1.8 to 1.17, but not Java 2.0 (after disablement occurs). Obviously removing SecurityManager infrastructure from OpenJDK is a significant undertaking requiring significant resources to audit the impacts of removal of Permission checks, unfortunately we don't have these resources and the alternative methods of putting alternative access controls in place isn't obvious and will require research, as you state below, I know very little about how this can be done without using existing Java access controls. I think during this transition period, while developers are reestablishing what constitutes secure programming, there will be many new bugs exposed that SecurityManager was obscuring. Personally, I will be observing from the sidelines to learn new security features and best practices I'm not yet aware of. It's always best to wait and see the outcome of experiments before diving in. The problem with Security is many developers don't like it, or want it to be someone else's problem, so we will always have reasons to remove whatever Security API's are added to Java because of limited use. Perhaps I'm conflating security with access control again? -- Regards, Peter On 14/05/2021 2:15 am, Sean Mullan wrote: On 5/12/21 5:41 PM, Peter Tribble wrote: Let me give a concrete example: Parsing and rendering a PDF file that may contain references to fonts or other resources. We know exactly where the files are installed, so wish to allow the rendering routine access to the fonts it will need. But not to any other files, and not (normally) to network resources at all. Note that we trust the code, but not necessarily the document it's parsing. (Although the document itself may be perfectly well formed - document formats often allow embedding references to 3rd-party objects, undesirable as that may be.) There are a range of such issues in document parsing and rendering. And unfortunately, the good libraries for this task are proprietary so we can't modify them to apply the restrictions we're after. The (server-side) application does need access to files and network resources at other times; it's only when it goes into the rendering step that we lock it down, and unlock it once done. I know very little about this area, but I would think a good PDF rendering library would include security features which prevent arbitrary file/network access from untrusted documents by default or at least give you the ability to control that. --Sean
Re: JEP411: Missing use-case: Monitoring / restricting libraries
So it targets 17. Java 8 has planned support to 2030, longer than 17, it seems unlikely the SecurityManager will still be present in an LTS release later than 17, given that 11 was the last, maybe 23 will be the next LTS version. Of course these aren't OpenJDK considerations, maybe someone will decide to support a version of Java that has it longer, or maybe they wont. It would be nice to have certainty about which release it will be removed from, for planning purposes. Again it would seem that this isn't a consideration of OpenJDK. Is there an OpenJDK community project group that maintains older Java versions I can join? Regards, Peter. On 13/05/2021 6:21 pm, Ron Pressler wrote: Whatever version JEP 411 targets, it will *not* remove the Security Manager. Even if 411 targets 17, the Security Manager will still be in 17, precisely because of the deprecation policy intended to give people time to adjust. — Ron On 13 May 2021, at 01:44, Peter Firmstone wrote: Ron, Can JEP 411 be targeted against Java 18 please? I realize long term support is not OpenJDK's concern, however other's are planning Java 17 to be a long term support release and that will impact us. Thank you, Peter Firmstone Zeus Project Services Pty Ltd. On 13/05/2021 7:43 am, Ron Pressler wrote: On 8 May 2021, at 05:55, Peter Firmstone wrote: What would help in future: • Define a core Java api, a javadoc annotation? If parts of it are deprecated, they will not be removed for eg 3 LTS releases, pick a number, it provides certainty. Developers writing new software then know if they use this api, they will not be harmed by breaking changes for x years. • Removal of api from java.* or javax.* are breaking changes. This is worse than a library breakage, as we can write a compatibility layer for a library. In my own software I provide a compatibility library for older versions of software written for Jini, it just decorates old api over new as a compatibility layer. For example we could write a compatibility layer for AccessController and doPrivileged methods, so they still work, without shotgun surgery. but we can't do that because it's in Java package namespace. • An annotation will let us know that we can write programs, without risk of incurring potentially significant technical debt. OpenJDK does not have the concept of LTS, and certainly not of "LTS releases.” Long-term support is a general term for services that anyone is free to offer for any OpenJDK version for any length of time, and even retroactively. You can choose to provide LTS to JDK 10 tomorrow if you like. OpenJDK does have a removal policy: https://openjdk.java.net/jeps/277. Its goal is to do exactly what you describe, and that policy is in effect for this JEP as it is for all other similar ones. But please read JEP 411 carefully. It does give you time; it does talk of a gradual process. • Sun always gave us plenty of time to remove usages of deprecated methods, it always took years to clean these up, but there are compiler warnings etc. My point is, we always got them removed eventually, meanwhile we were also able to take advantage of new features. This is the second time you’ve brought up Sun, as if it’s some disjoint group of people. People might have decided to change their policies due to changing circumstances, and the circumstances 15 years ago were certainly no identical to today. It appears to me that stack walking, which you singled out as a performance problem (it isn't), is likely causing difficulties for another project you're working on, which is why you are strongly motivated to see it removed. We would like to see it removed because we believe that the total good the Security Manager does the Java community as a whole does not appear to justify its high cost. In other words, we’d like to see it removed because we believe doing so would do more good for more people than not removing it. This will inflict pain on many projects, I just can't see people upgrading their software. Who's going to pay for all the hours of programming to convert perfectly good running code to a new api, just to upgrade to a newer version of Java? Removing stuff absolutely causes pain. We know that, we sympathise, and we’re trying to minimise it. But you have to understand that *not* removing stuff also causes pain, and not diverting resources elsewhere might dissuade other people from using Java because *their* needs aren’t addressed. We would have loved to please everyone but it’s impossible, so we have to make decisions, and whatever decision we make, someone will experience pain in the form of more work they have to do. We have to consider the total pain vs. total good over the entire Java ecosystem. Please also understand our perspective: you’re asking us to pay for hours of work to maintain something that fe
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Ron, Can JEP 411 be targeted against Java 18 please? I realize long term support is not OpenJDK's concern, however other's are planning Java 17 to be a long term support release and that will impact us. Thank you, Peter Firmstone Zeus Project Services Pty Ltd. On 13/05/2021 7:43 am, Ron Pressler wrote: On 8 May 2021, at 05:55, Peter Firmstone wrote: What would help in future: • Define a core Java api, a javadoc annotation? If parts of it are deprecated, they will not be removed for eg 3 LTS releases, pick a number, it provides certainty. Developers writing new software then know if they use this api, they will not be harmed by breaking changes for x years. • Removal of api from java.* or javax.* are breaking changes. This is worse than a library breakage, as we can write a compatibility layer for a library. In my own software I provide a compatibility library for older versions of software written for Jini, it just decorates old api over new as a compatibility layer. For example we could write a compatibility layer for AccessController and doPrivileged methods, so they still work, without shotgun surgery. but we can't do that because it's in Java package namespace. • An annotation will let us know that we can write programs, without risk of incurring potentially significant technical debt. OpenJDK does not have the concept of LTS, and certainly not of "LTS releases.” Long-term support is a general term for services that anyone is free to offer for any OpenJDK version for any length of time, and even retroactively. You can choose to provide LTS to JDK 10 tomorrow if you like. OpenJDK does have a removal policy: https://openjdk.java.net/jeps/277. Its goal is to do exactly what you describe, and that policy is in effect for this JEP as it is for all other similar ones. But please read JEP 411 carefully. It does give you time; it does talk of a gradual process. • Sun always gave us plenty of time to remove usages of deprecated methods, it always took years to clean these up, but there are compiler warnings etc. My point is, we always got them removed eventually, meanwhile we were also able to take advantage of new features. This is the second time you’ve brought up Sun, as if it’s some disjoint group of people. People might have decided to change their policies due to changing circumstances, and the circumstances 15 years ago were certainly no identical to today. It appears to me that stack walking, which you singled out as a performance problem (it isn't), is likely causing difficulties for another project you're working on, which is why you are strongly motivated to see it removed. We would like to see it removed because we believe that the total good the Security Manager does the Java community as a whole does not appear to justify its high cost. In other words, we’d like to see it removed because we believe doing so would do more good for more people than not removing it. This will inflict pain on many projects, I just can't see people upgrading their software. Who's going to pay for all the hours of programming to convert perfectly good running code to a new api, just to upgrade to a newer version of Java? Removing stuff absolutely causes pain. We know that, we sympathise, and we’re trying to minimise it. But you have to understand that *not* removing stuff also causes pain, and not diverting resources elsewhere might dissuade other people from using Java because *their* needs aren’t addressed. We would have loved to please everyone but it’s impossible, so we have to make decisions, and whatever decision we make, someone will experience pain in the form of more work they have to do. We have to consider the total pain vs. total good over the entire Java ecosystem. Please also understand our perspective: you’re asking us to pay for hours of work to maintain something that few use, hours that could go into work more people could enjoy. — Ron
Re: Performance differences between Java 8,, 11, 14 and 16
Hi Alan, Understood, not sure if I could give AccessController that kind of workout using JDK classes alone, the library code is very performant. It might be possible if AllPermission is granted and ClassLoaders are created using only class bytes, without accessing files, to avoid using the Java's PolicyFile, URLClassLoader, CodeSource and URL. To be honest though, it would be a reasonable size task, sorry, but I don't have enough time. Mahalo is a network transaction manager service, in the test it runs through 1000 transactions, each with 1 to 9 network participants, each participant is a proxy with a unique ProtectionDomain and ClassLoader (represents Server identity in the client) that's making remote method calls over TLS connections. By the time the test complete's it's created over 20,000 ProtectionDomains and over 5,000 ClassLoader's, not all used at the same time of course, but there are a number of codebases involved, so there could be around 10 to 20 ProtectionDomain's in a call stack at the same time, these are being replaced, so it gives the AccessController quite a good workout. Note these ProtectionDomains run using the principle of least privilege. Even TLSv1.3's overhead is un-noticable, it's basically only limited by the speed of the underlying network sockets. Thank you to the developers who made TLSv1.3 run so well. https://imgur.com/VcSwffC https://imgur.com/VcSwffC https://imgur.com/hmFPnAW https://imgur.com/IcUXzNK https://imgur.com/MutdNNt -- Regards, Peter Firmstone Zeus Project Services Pty Ltd. https://imgur.com/VcSwffC https://imgur.com/VcSwffC On 12/05/2021 6:00 pm, Alan Bateman wrote: On 12/05/2021 07:18, Peter Firmstone wrote: https://github.com/pfirmstone/JGDMS/blob/trunk/qa/src/org/apache/river/test/impl/mahalo/RandomStressTest.java https://github.com/pfirmstone/JGDMS/blob/trunk/qa/src/org/apache/river/test/impl/mahalo/RandomStressTest.td It would be great if there were a JMH benchmark that didn't have any dependency on Apache River or other libs. That would make it a bit easier to study/diagnose. -Alan
Re: Performance differences between Java 8,, 11, 14 and 16
Steps to reproduce: Use git to checkout https://github.com/pfirmstone/JGDMS.git Set the following path variables to your Java installation directory: JAVA_HOME=file path to java 8 JDK_HOME=file path to jdk 8 (same as above) Make sure the java command returns Java 8. $java -version Change directory into JGDMS $mvn package Change directory into qa: cd ../qa Create a file called build.properties Add the following 2 lines to build.properties (replace with your path): river.home=C:/Users/peter/Documents/NetBeansProject/JGDMS run.tests=org/apache/river/test/impl/mahalo/RandomStressTest.td Then build the qa suite: $ant build Then edit the RandomStressTest.td file and add the following line (for repeatability) org.apache.river.test.impl.mahalo.RandomStressTest.seed=1620791630932 Then change JAVA_HOME and JDK_HOME to your preferred version of java, and change your path to point to the java version you want to test. Then run the test: $ant run-tests -- Regards, Peter Firmstone Zeus Project Services Pty Ltd. On 12/05/2021 4:18 pm, Peter Firmstone wrote: 你好先生范, 不客气, https://github.com/pfirmstone/JGDMS/blob/trunk/qa/src/org/apache/river/test/impl/mahalo/RandomStressTest.java https://github.com/pfirmstone/JGDMS/blob/trunk/qa/src/org/apache/river/test/impl/mahalo/RandomStressTest.td With the follow property set to reproduce: org.apache.river.test.impl.mahalo.RandomStressTest.seed=1620791630932 谢谢 -- Regards, Peter Firmstone Zeus Project Services Pty Ltd. On 12/05/2021 3:42 pm, Xuelei Fan wrote: Hi Peter, For further understanding, may I know more details about the test code? Thanks, Xuelei On May 11, 2021, at 10:31 PM, Peter Firmstone wrote: Just ran some performance stress tests to compare performance between Java 8 and Java 14 with SecurityManager enabled and using TLSv1.3 secure communications. On Java 8, AccessController.getStackAccessControlContext[native]() is 7,741ms of process CPU consumption. On Java 11, AccessController.getStackAccessControlContext[native]() is 7,769ms of process CPU consumption. On Java 14, AccessController.getStackAccessControlContext[native]() is 133,815 ms of process CPU consumption. On Java 16, AccessController.getStackAccessControlContext[native]() is 5,144 ms of process CPU consumption. I wasn't expecting such a huge difference in performance. Total time for the test on Java 8 is 97 seconds. Total time for the test on Java 11 is 1 minute 37 seconds (same as Java 8) Total time for the test on Java 14 is 2 minutes and 1 second. Total time for the test on Java 16 is 1 minute and 36 seconds. Both tests are using TLSv1.3 secure communications. It would seem that Java 16, 8 and 11 are the performance winners in this case, where the above method is less than 0.5% of CPU. Java 14 is around 15% of CPU. Not sure why Java 14 is performing poorly in comparison to other versions, the results are pretty consistent to the second. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: Performance differences between Java 8,, 11, 14 and 16
你好先生范, 不客气, https://github.com/pfirmstone/JGDMS/blob/trunk/qa/src/org/apache/river/test/impl/mahalo/RandomStressTest.java https://github.com/pfirmstone/JGDMS/blob/trunk/qa/src/org/apache/river/test/impl/mahalo/RandomStressTest.td With the follow property set to reproduce: org.apache.river.test.impl.mahalo.RandomStressTest.seed=1620791630932 谢谢 -- Regards, Peter Firmstone Zeus Project Services Pty Ltd. On 12/05/2021 3:42 pm, Xuelei Fan wrote: Hi Peter, For further understanding, may I know more details about the test code? Thanks, Xuelei On May 11, 2021, at 10:31 PM, Peter Firmstone wrote: Just ran some performance stress tests to compare performance between Java 8 and Java 14 with SecurityManager enabled and using TLSv1.3 secure communications. On Java 8, AccessController.getStackAccessControlContext[native]() is 7,741ms of process CPU consumption. On Java 11, AccessController.getStackAccessControlContext[native]() is 7,769ms of process CPU consumption. On Java 14, AccessController.getStackAccessControlContext[native]() is 133,815 ms of process CPU consumption. On Java 16, AccessController.getStackAccessControlContext[native]() is 5,144 ms of process CPU consumption. I wasn't expecting such a huge difference in performance. Total time for the test on Java 8 is 97 seconds. Total time for the test on Java 11 is 1 minute 37 seconds (same as Java 8) Total time for the test on Java 14 is 2 minutes and 1 second. Total time for the test on Java 16 is 1 minute and 36 seconds. Both tests are using TLSv1.3 secure communications. It would seem that Java 16, 8 and 11 are the performance winners in this case, where the above method is less than 0.5% of CPU. Java 14 is around 15% of CPU. Not sure why Java 14 is performing poorly in comparison to other versions, the results are pretty consistent to the second. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Performance differences between Java 8,, 11, 14 and 16
Just ran some performance stress tests to compare performance between Java 8 and Java 14 with SecurityManager enabled and using TLSv1.3 secure communications. On Java 8, AccessController.getStackAccessControlContext[native]() is 7,741ms of process CPU consumption. On Java 11, AccessController.getStackAccessControlContext[native]() is 7,769ms of process CPU consumption. On Java 14, AccessController.getStackAccessControlContext[native]() is 133,815 ms of process CPU consumption. On Java 16, AccessController.getStackAccessControlContext[native]() is 5,144 ms of process CPU consumption. I wasn't expecting such a huge difference in performance. Total time for the test on Java 8 is 97 seconds. Total time for the test on Java 11 is 1 minute 37 seconds (same as Java 8) Total time for the test on Java 14 is 2 minutes and 1 second. Total time for the test on Java 16 is 1 minute and 36 seconds. Both tests are using TLSv1.3 secure communications. It would seem that Java 16, 8 and 11 are the performance winners in this case, where the above method is less than 0.5% of CPU. Java 14 is around 15% of CPU. Not sure why Java 14 is performing poorly in comparison to other versions, the results are pretty consistent to the second. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: JEP411: Restricting/logging library usages using a SecurityManager
Just some references regarding Roel's original argument below: https://techbeacon.com/security/third-party-libraries-are-one-most-insecure-parts-application https://debricked.com/blog/2021/01/02/vulnerabilities-in-dependencies/ https://www.tripwire.com/state-of-security/vulnerability-management/managing-security-risk-introduced-by-third-party-libraries/ https://www.infosecurity-magazine.com/opinions/third-party-libraries-the-swiss/ https://auth0.com/blog/third-party-assets-security-risks-management/ It will be interesting to see how OpenJDK plans to develop new ways to assist developers to mitigate third party library risks. -- Regards, Peter Firmstone 0498 286 363 Zeus Project Services Pty Ltd. On 16/04/2021 7:10 am, Roel Spilker wrote: Hi all, So I do get why RMI and Applets are no longer used. I also agree that the performance and usability of the current implementation is suboptimal, and that configuring the security manager through text files is also not that easy. But on my server application, we use libraries. And I'm very interested on how they behave. I would like to log or restrict the following actions: - Spawning new processes - Unexpected file access - Unexpected network traffic Currently, our application sets a custom written security manager to restrict or log those aspects. For example, we would block all XXE attacks by just having our security manager. In JEP411 I did not find a way to do those things without a security manager. What does the security group think about these use cases? Does it still make sense to deprecate/remove the entire security manager? Would a replacement for certain concerns be in order? Kind regards, Roel Spilker
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Ron, Thanks for the discussion. Although we have different opinions, I do appreciate that you took the time to reply. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 7/05/2021 1:17 pm, Peter Firmstone wrote: On 6/05/2021 9:46 pm, Ron Pressler wrote: That is correct. Here is where this is mentioned for ForkJoinPool: https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/util/concurrent/ForkJoinPool.html And here it is for virtual threads (the JavaDoc is still a WIP): https://download.java.net/java/early_access/loom/docs/api/java.base/java/lang/Thread.html#startVirtualThread(java.lang.Runnable) Can I use doPrivileged blocks with context, or will they be innefective? If yes to the former and no to the latter, then it's great these threads have no permission by default. Just trying to work out if I can use it with secure network connections? -- Regards, Peter
Re: JEP411: Missing use-case: Monitoring / restricting libraries
What would help in future: 1. Define a core Java api, a javadoc annotation? If parts of it are deprecated, they will not be removed for eg 3 LTS releases, pick a number, it provides certainty. Developers writing new software then know if they use this api, they will not be harmed by breaking changes for x years. 2. Removal of api from java.* or javax.* are breaking changes. This is worse than a library breakage, as we can write a compatibility layer for a library. In my own software I provide a compatibility library for older versions of software written for Jini, it just decorates old api over new as a compatibility layer. For example we could write a compatibility layer for AccessController and doPrivileged methods, so they still work, without shotgun surgery. but we can't do that because it's in Java package namespace. 3. An annotation will let us know that we can write programs, without risk of incurring potentially significant technical debt. 4. Sun always gave us plenty of time to remove usages of deprecated methods, it always took years to clean these up, but there are compiler warnings etc. My point is, we always got them removed eventually, meanwhile we were also able to take advantage of new features. If OpenJDK does that, then I personally would continue use of Java for new projects. OpenJDK is not doing anything to ease the pain of migration, when you could do better. I'm curious, what are other developers using instead of Subject.doAs, JAAS and AccessController.doPrivileged to authenticate TLS connections? In JGDMS I found: 916 matches of AccessController in 264 files, and that's excluding tests. 515 matches of AccessControlContext in 121 files, again excluding tests. It appears to me that stack walking, which you singled out as a performance problem (it isn't), is likely causing difficulties for another project you're working on, which is why you are strongly motivated to see it removed. This isn't like other API's that are being removed, so far we have only been impacted in parts of our codebase. This will inflict pain on many projects, I just can't see people upgrading their software. Who's going to pay for all the hours of programming to convert perfectly good running code to a new api, just to upgrade to a newer version of Java? Just can't see it happening. -- Regards, Peter On 8/05/2021 8:42 am, Ron Pressler wrote: Many of the people who worked at Sun still work at Oracle on Java today, and that group includes all the people who signed their names on this JEP, but Java today has ten more years of baggage to maintain than it did back then. The speed at which things are removed after deprecation is meant to be proportionate to their level of use. I believe that, despite the fact that some less used things are, indeed, being removed now while they weren’t before, Java still compares favourable with many other languages/platforms as far as stability and compatibility is concerned. I think that saying that “move fast and break things” is the new philosophy is not only unfair, but very, very far from the truth. — Ron On 7 May 2021, at 23:20, Peter Firmstone wrote: On 8/05/2021 4:21 am, Ron Pressler wrote: Deprecation/removal JEPs, and this one is no exception, make the following claim: that the total good a certain JDK capability currently contributes to the Java ecosystem at large does not justify the cost of its maintenance, and it should, therefore, be removed — gradually, of course, and with enough time for users to find alternatives. An argument against such a JEP would take the form of, no, the total good the feature contributes to the ecosystem does justify its cost. Sun Microsystems would deprecate, but were very slow to remove API that caused breakages, they were also very considerate about how they would modify api in a backward compatible manner. OpenJDK has already demonstrated it removes api very quickly after deprecation. OpenJDK has adopted the move quickly and break things philosophy. I really like the new language features under development, but the continual breakages are causing me to rethink. I still haven't worked out how to replace some of the more recently removed features, we are still building on Java 8, because of missing components in Java 11, although we use features from and test on later versions. We haven't been testing on Java 8, because our default ciphers target the most recent versions and we disable anything less by default. Other breaking changes that have been removed can be replaced by library code, but cause breakages since we are unable to use the java.* package namespace. It would be friendlier, if OpenJDK allowed libraries to be developed separately, using the java.* namespace, perhaps as part of the project. This core platform feature that will be removed, pr
Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 8/05/2021 4:21 am, Ron Pressler wrote: Deprecation/removal JEPs, and this one is no exception, make the following claim: that the total good a certain JDK capability currently contributes to the Java ecosystem at large does not justify the cost of its maintenance, and it should, therefore, be removed — gradually, of course, and with enough time for users to find alternatives. An argument against such a JEP would take the form of, no, the total good the feature contributes to the ecosystem does justify its cost. Sun Microsystems would deprecate, but were very slow to remove API that caused breakages, they were also very considerate about how they would modify api in a backward compatible manner. OpenJDK has already demonstrated it removes api very quickly after deprecation. OpenJDK has adopted the move quickly and break things philosophy. I really like the new language features under development, but the continual breakages are causing me to rethink. I still haven't worked out how to replace some of the more recently removed features, we are still building on Java 8, because of missing components in Java 11, although we use features from and test on later versions. We haven't been testing on Java 8, because our default ciphers target the most recent versions and we disable anything less by default. Other breaking changes that have been removed can be replaced by library code, but cause breakages since we are unable to use the java.* package namespace. It would be friendlier, if OpenJDK allowed libraries to be developed separately, using the java.* namespace, perhaps as part of the project. This core platform feature that will be removed, probably after Java 17, but before the following long term support version cannot be replaced by a library. The maintenance debt is building up too fast to keep up with. I can't justify writing new projects in Java until the API has stabilized, it's fair to say the new API is Java like, but C# is also Java like, as is Android. It's clear OpenJDK wants Java to be like younger languages, and since that's where it's headed, I might as well select one of those instead, what kept me developing on Java was its stability and performance, when newer languages could do the same with less. Performance of newer languages will improve with time, just like Java did and their API's will become more stable. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 6/05/2021 9:46 pm, Ron Pressler wrote: Trying to convince people, at this point, after twenty five years that the Security Manager isn’t complicated after all might be too little too late. Static policy, terrible performance, no scalability at all, and the fact that you continually have to edit policy files manually, and there's no auditing tools? Ha ha ha. :) It's complicated, ha ha ha, it hardly works! Why would a developer spend time writing concurrent code, then turn on security and slow their hard work to a crawl? That's why they simplify it and bypass the policy. No, complexity is not the problem. It was a good design for 1997, but the java code it's written in is also from 1997 with little maintenance since. For shame. From my observations, the native code in AccessController is scalable and performant and has little overhead, someone has done some very good work there, that has to be more recent. This is a very good piece of work, very good indeed. Sorry, I had to point out some truths. My static policy (as stated previously there is a dynamic policy also) is a direct drop in replacement, you could ship with that, it would be a start. You could even remove the Java policy implementation and I can make my policy implementation available on Maven. It's AL2.0 licensed, I did look at donating it some time ago. The code has provenance, I'm not the sole author, I can only donate parts of it under GPL2.0 I can also donate the profiling tool. The thing is, if it was performant, people would stop switching it off, and if there were tools to handle policy complexity, then they will start using it, they have to use it for Principal permissions. Here's what OSGi does, they associate permissions with modules, to reduce complexity: https://docs.osgi.org/specification/osgi.core/8.0.0/service.condpermadmin.html -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: [External] : Re: JEP411: Missing use-case: Monitoring / restricting libraries
relying on different code having different permissions *in the same process* is not a good strategy. After twenty five years of Security Manager with little use and less still proper use, and with most of the ecosystem not supporting this mechanism anyway so even if improvement were justified it would take years until people could enjoy it, increasing investment in it would be, in my opinion, throwing good money after bad. Experience with poor or incomplete implementations has shown that. You have to admit then, that you have zero experience with a good implementation. It's just access control, you are mistakenly still thinking about sandboxes. In the sandbox model, the sandbox is the be all and end all of security, the sandbox is supposed to present a safe and impenetrable api, in the cheese model, access control is just one layer of cheese with holes. Because nothing is perfect, it's better to have multiple layers of security, as it reduces the chances of things like gadget chains succeeding, rather than a be all and end all wack-a-mole whitelist approach that seems to be popular of late. https://securityandpeople.com/2017/07/human-errors-in-cyber-security-a-swiss-cheese-of-failures/ Anyone can use JGDMS to enjoy it now, little investment is required, the work is done. It also takes the pain out of implementing TLS communications as well, including preserving the Subject across JVM's. https://github.com/pfirmstone/JGDMS It supports Maven and OSGi, so you can dynamically download libraries and automatically assign permissions. Haven't implemented support for Java modules yet, as we still compile on Java 8, although funnily enough we use later TLSv1.3 cypher suites. Oh, that reminds me OSGi also implements SecurityManager and has dynamic permissions, it can revoke as well as grant permissions, are you aware of it? It uses a different mechanism though, because it pre-dates Java 1.4, I don't recall the implementation. It's a shame it took SecurityManager's proposed removal for you to discover it has practical application. You are right on one respect, not enough people take security seriously. Security and the Security Manager are not the same thing, Agreed, but your statement is not relevant to the points I'm making here, you are still thinking about sandboxes, instead of access control. Sandbox whitelists is wack-a-mole, so is white listed Serialization without atomic failure input validation. Maybe it's not that people don't take security seriously, it's that they think white lists are an appropriate form of security, without authentication and validation, they are not. Same problem with the sandbox. Access Control is still useful, when it's not treated as the front line of security, but a more rearward layer that is expected to have holes, because it has limitations, so we instead depend on a armory of defense layers, each with flaws, that we try to manage. The Cyphers of the late 90's are no longer in use, and neither is the sandbox. virtually none of the security work in the JDK in the past few years took place in the Security Manager, Because Serialization is not failure atomic, is performed by privileged system code and developers had open network ports without authentication that an attacker could intercept and use gadget chains to take over the JVM, I don't see how this is relevant to access control for trusted code and trusted users. There were similar problems with XML. This is why whitelists and whak-a-mole is not security either, also why I think whitelist serialization will come back to bite again, because people using whitelists think they can process data, without proper input validation. What makes it works is it's a separate concern, it breaks object encapsulation. There should be a command line parameter to disable Serialization, in such a way that it cannot be enabled at runtime. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Thanks Alan, I understand the motivation. The front line of security is authentication, privacy (encryption), verification and validation with failure atomicity. SecurityManager is unfortunately named, giving the impression that it has responsibility for security. In truth, it's ONLY an access decision tool, rather than a controller or manager. It can have benefits for security, when employing the principle of least privilege, but it is not in and of itself a solution for system security, as you most rightly imply, it becomes an expensive game of whack-a-mole. I think the problem is one of perception. I would like to propose instead of deprecation, is that we simply document that security manager no longer performs it's original intended role, that it's role today is only to assist in access decisions. Lets remove the concept of a SecurityManager sandbox, let's put in big bold friendly letters, "THIS IS NOT A SANDBOX, NOR SHOULD IT BE RELIED UPON TO SECURE YOUR SYSTEM, IT IS ONLY SUITABLE FOR ASSISTING IN ACCESS DECISIONS FOR TRUSTED USERS AND CODE" And also explain why, ie the game of whack-a-mole. Then a discussion of least privilege principles would be appropriate. In your examples below, where someone has forgotten to check a permission, or preserve context across threads, these are not exploitable until the front line of defense fails. Typically forgetting to preserve the context prevents a TLS connection from opening and it will be reported and fixed, but it is not a security risk, it just means you can't connect until it's fixed. Often the failure to check a permission is benign, because the code is trusted, but it does provide a way to manage who has access to data and to monitor it. Java Serialization shouldn't be used to process un-trusted data either, nor should it be used over an unencrypted connection, even with a whitelist, there is no guarantee of security, as it lacks validation failure atomicity. So how about we lower the bar for reporting, such that a bug in access control is treated differently than bugs that truly threaten security? Lets make some rules around that so that when someone reports a bug, it's not escalated to a security bug if one of the front line defenses would prevent an untrusted party taking control of the jvm. Create a new bug category called ACCESS CONTROL BUGS, still fix holes in access control when reported, but don't escalate it to a level of security that no one can see, instead, publish the bug and let people submit patches. This will reduce the maintenance burden and cost. We still want access control to function, but recognize its limitations. At work in your typical corporate environment, users are given access to directory's and files using Kerberos / RBAC and are typically disallowed from installing an application on their computer and have to call up the IT help desk to have software installed, this is because your average user will install anything onto their computer to get things done. The average user isn't a hacker, that's going to hack your systems. Access control on Java should be treated similarly, I install this code, or authorize this user, on my JVM but I'm going to place some limitations on what they are allowed to do. We are not expecting them to hack the system. I found the type of bug you are talking about myself only recently: https://github.com/pfirmstone/JGDMS/issues/125 Privileged methods and context are commonly used to establish TLS connections, so removing AccessControlContext and AccessController will break this code, when we already have the tools we need to establish TLS connections. Save the work of writing new tools to replace existing functionality, document the existing, change how we respond and devote the resources to something more important. That way our existing code should also remain functional, so it's win-win. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd. On 6/05/2021 9:48 pm, Alan Bateman wrote: On 06/05/2021 11:26, Peter Firmstone wrote: OpenJDK seems to have assumed that no one was using SecurityManager based on one research report. I don't think this is right. Instead I would say that many of us have rarely encountered deployments on the server-side that are using a SecurityManager to enforce security as envisaged by the Java security model. I've been at Java conferences where the sessions on this topic had less than 10 people in the room. Most of the actual usages that I've come across have been more like using the security manager as a convenient way to intercept network and file access for the purposes of logging or blocking. These usages may not have a need for protection domains, stack walks, policy files and the other complexity that comes with the security model. One thing that is missin
Re: JEP411: Missing use-case: Monitoring / restricting libraries
ileges? There is some documentation on the common pool regarding SecurityManger, but little on Loom's virtual threads. Can someone point me to a reference to security documentation on these features please? Security Manager is not a central component for securing trusted server-side code, a good thing, too, because few systems use it and it doesn’t defend against some of the most common and dangerous exploits. In fact, the Security Manager is already crippled on the common pool <https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/util/concurrent/ForkJoinPool.html> (and on Loom’s virtual threads <https://download.java.net/java/early_access/loom/docs/api/java.base/java/lang/Thread.html#startVirtualThread(java.lang.Runnable)>) because setting up appropriate security contexts would defeat the performance requirements of those constructs, and using it with |CompletableFuture|s or any asynchronous (or “reactive”) context requires the developer to carefully capture and reestablish <https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/security/AccessController.html> security contexts as operations travel from one thread to another. If we disable Java serialization, we don't use XML, we validate invariants in our constructors and design for failure atomicity, we design to encapsulate, we avoid shared mutable state as much as possible, use immutability, we check third party libraries for security vulnerabilities, we perform static code analysis and we are running code using the principle of least privilege, we use the latest available TLSv1.3 cyphers and disable earlier versions. Do you believe we are still vulnerable to these most common and dangerous exploits? Perhaps we are still vulnerable to unknown or zero day exploits? It is my opinion that removing the principle of least privilege will not improve the security of our software, but rather degrade it. But then Java hasn't had a good reputation for security in recent years, thanks to Serialization, granted OpenJDK has done a lot to swat bugs as they're discovered in recent years, it will take a little more time to build a good reputation. Personally I wouldn't be removing SecurityManager, I'd be addressing the issues and cleaning it up so programs can be run with principles of least privilege, I know it's not perfect, but nothing ever is, but it should improve with time, if it's not neglected and replaced if needed. https://en.wikipedia.org/wiki/Principle_of_least_privilege Maybe Unix sockets will provide a performance boost, Sockets are our major hotspot at 42.5% https://pfirmstone.github.io/JGDMS/old-static-site/images/River-internet_mahalo_randomStressTest.PNG I guess you now have a use case for what SecurityManager is capable of. It's a shame it took SecurityManager's proposed removal for you to discover it has practical application. You are right on one respect, not enough people take security seriously. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 5/05/2021 10:55 pm, Sean Mullan wrote: -bcc jdk-dev -cc security-dev On 5/5/21 12:04 AM, Peter Firmstone wrote: I think we are talking past each other here. You keep talking about untrusted code, which sounds like applets to me. I've read and still have a copy of Li Gong's book, applets were only one of the considerations. I am talking about authorization and access control. We use and develop distributed or p2p systems, we don't allow untrusted code to run at all, never ever, that's a dumb idea, so lets stop talking about untrusted code, we don't use that. We do utilize dynamic downloaded code from others and use dynamic class loading, we verify this prior to loading. We check it's authorized to run before running it. Again I repeat, we do not run untrusted code, that would allow an attacker to cause denial of service etc, the JVM has no control over badly behaving code. But you use self-signed certificates to sign the code that will be run. There is no trust in self-signed certificates unless you have previously used some out-of-band mechanism to trust the public key. I still don't understand why this is not the same as running untrusted code, even if the code is sandboxed. And trusting the TLS server is not an equivalent basis of trust. Yes, we have a dynamic out of band mechanism. We authenticate the other party using TLS, and depending on who it is (whether Permission has been granted to their Principal), we dynamically grant them permission to load their code on our system. They can use a self signed certificate on their code, and we dynamically grant permission for the code signed by that certificate to be loaded, otherwise they can use a cryptographic checksum, so we can be sure their code hasn't been modified by a third party, it just simplifies the process, so we don't have to introduce another CA, we are actually trusting the remote end to audit their own code because we know who they are (configuration concerns). We also give that code Permission to connect to the remote end of the TLS connection, because threads only run with one Subject, we need the code to represent the remote Principal. We are not concerned with the names of classes or packages in their code as that is their implementation concern. Provided there is no data theft, if something is taken down by badly behaving software, it has a level of fault tolerance, services will be restarted automatically, if they fall over, they are re-activated. We don't use RMI's activation implementation, but we do depend on some activation API classes. The problem with removal of access control is we would be permitting unrestricted access to a trusted third party, who doesn't require unrestricted access, nor is it in our interests to allow it. It's a matter of trusting the TLS enpoint, both clients and servers are authenticated. But these may just be servers talking to each other, not necesarilly a client server relationship, for example a client of a service may require an event notification, so it passes a Remote object as a parameter to the service, now the server of the service, is also a client, when it sends the event notification to listeners. It's distributed, p2p. This is used by some as a cluster back end for JEE, although I must admit I don't know many of the details there. One of the reasons SecurityManager didn't control many Java Serialization vulnerabilities, is because ObjectInputStream was granted AllPermission as it was a platform class. Clearly ObjectInputStream belonged in an unprivileged domain. Also it's not good that Java serialization circumvents invariant checks that constructors perform. Our software also allows the company we are dealing with for example, to provide their own GUI window within a GUI so to to speak, to allow a supplier to be integrated in to the system, as an example. That us if it's a system with user interaction, otherwise it could be an automated process between two systems, based on an agreed interface. The only thing known prior are the Java interfaces used for intra system communication, the platform software (which we try to maintain as backward compatible) and the principal of the other party which we have granted permission to. Also I have been developing a public api for serialization (we've discussed the de-serialization component of it previously), which is suitable for other serialization protocols as well as Java's. We don't support circular links in serialized object graphs. The only class we found that required a circular link was Throwable and we have found it's possible to program around circular links. Developers implementing it use an annotation and implement a public constructor for de-serialization, a public static method for serialization and another public stat
Re: JEP411: Missing use-case: Monitoring / restricting libraries
On 5/05/2021 10:08 am, Ron Pressler wrote: I wouldn’t say Java (or anything else, for that matter) is “able" to do it now, except in the sense that people (scientists) are able (in a billion-dollar particle accelerator) to transmute lead into gold (a few atoms). We’ve had twenty five years to convince the world this could work, the world isn’t buying, and our job isn’t to sell ideas but to serve millions of developers by giving them what we believe they need now, not what we wished they wanted. — Ron We just want our software to work Ron, we invest years of time and effort, we just want it to work. We don't want to have to test and rework it for every Java release, you are creating too much maintenance for us to keep up with. You'll be serving fewer and fewer developers as more and more are left behind as breakages accumulate. I was at least keeping up and testing newer releases, even though we still only build on Java 8. Last I checked the stats, 58% were using Java 8, 23% using Java 11 and 6% of developers using 12 or newer. I think you'll have trouble selling it as you say, we won't have time to learn and implement new language features if we're too busy fixing breakages. Hard life creates hard people, hard people create easy life, easy life creates soft people, soft people create hard life. -- Regards, Peter Firmstone My personal opinion only.
Re: JEP411: Missing use-case: Monitoring / restricting libraries
A VALUABLE LESSON FOR ANY JAVA DEVELOPER: DON'T PUBLISH ANY java.* package namespace API'S THAT MAY BE AT RISK OF LATER REMOVAL IN YOUR API, java.* API's ONCE REMOVED CANNOT BE REPLACED. IF YOU ARE CONCERNED SOMETHING MAY BE REMOVED IN FUTURE, SUBCLASS IT IN YOUR API, OR CREATE AN INTERFACE WITH SUBCLASS DECORATOR, SO THAT YOU HAVE SOME CONTROL OVER BACKWARD COMPATIBLE API EVOLUTION. On 5/05/2021 10:08 am, Ron Pressler wrote: Resent with plain-text formatting (I hope) & corrections/rephrasing On 4 May 2021, at 03:49, Peter Firmstone wrote: Yes, I'm sure millions of developers don't use the security infrastructure because they only have low value data to protect, or it belongs to someone else and developers that do, can use it incorrectly, it's probably worse to do the latter, but then people synchronize incorrectly too, but we don't remove synchronization because of that. The Security Manager hasn’t been a central part of Java’s server-side security for years. Some of the most critical and best-secured systems in the world are written in Java, and they don’t use the Security Manager, so let’s not equate a particular sandboxing mechanism designed for untrusted code with security. Got any example best-secured systems? I think we are talking past each other here. You keep talking about untrusted code, which sounds like applets to me. I've read and still have a copy of Li Gong's book, applets were only one of the considerations. I am talking about authorization and access control. We use and develop distributed or p2p systems, we don't allow untrusted code to run at all, never ever, that's a dumb idea, so lets stop talking about untrusted code, we don't use that. We do utilize dynamic downloaded code from others and use dynamic class loading, we verify this prior to loading. We check it's authorized to run before running it. Again I repeat, we do not run untrusted code, that would allow an attacker to cause denial of service etc, the JVM has no control over badly behaving code. We've been using this since Java 1.4, to control access to networks, file systems and other custom permissions we created, it is used to protect access to sensitive data. We are still using it. I understand access control will be removed: https://docs.oracle.com/javase/8/docs/technotes/guides/security/spec/security-spec.doc4.html#20389 We would like to continue restricting access to data after the above is removed. Will Java be introducing new Role Based Access Control API or something similar? Our software will fail to run on Java after the above is removed. I understand we have to remove access control functionality from our software for it to continue functioning? We do need to understand how this will impact security, I think you are trying to convince me or yourself that security will not be impacted? We can't just assume we can remove access control and our software will be no less secure. What is the new API for access control in Java? Obviously we won't have a call stack with domains, I don't know how we will transfer the user Subject to other threads, for TLS and Kerberos connections. No doubt something is planned. Is the recommendation simply not to upgrade Java until new access control API is developed? Please provide some examples, migration options suggestions will be appreciated. I’ve jotted down some thoughts in a blog post: https://inside.java/2021/04/23/security-and-sandboxing-post-securitymanager/ Noted, a good start. I get the feeling this JEP is being rushed through however, with little regard for those of us who were foolish enough to use Java's security API's and will have to suffer the consequences. With Serialization, we've been given more than ample notice to do something about migrating away from it, but OpenJDK paints over it and wastes resources adding features to putty and paint over it some more, features that no one uses. Removing Serialization has greater appeal :) This step to remove SecurityManager is so sudden with no replacement options, it's a broken promise to developers, who've invested in Java. Removing SecurityManager has a significantly negative effect on security for me, just so you know. I'm not happy about its proposed removal, but I realise there's not much I can do about it, other than request it be done in the least painful manner. I began learning Java over 20 years ago, I understand the need to keep Java relevant, however move quickly and break things is for younger software platforms. Not everyone has to agree with every priority decision, and the process doesn’t require convincing every last Java developer. But it is not sudden, and there will be alternatives for those aspects of Security Manager that more people use. I don’t think it is fair to h
Re: Java Security: JEP: 411: Deprecate the Security Manager for Removal - What about Serialization?
Clarification inline below On 4/05/2021 8:35 am, Peter Firmstone wrote: On 4/05/2021 5:12 am, Sean Mullan wrote: -bcc jdk-dev -cc security-dev On 4/30/21 10:04 PM, Peter Firmstone wrote: In our software we use a ProtectionDomain to represent a remote server, because a thread only runs with the user's Subject (and that Subject must be carefully preserved for other threads), there is no way to represent the remote Server's Subject in a local domain , other than with a ProtectionDomain. Our software is peer to peer, clients can be servers and servers can also be clients. Code to interact with the server is downloaded via Maven and loaded. Any permission's granted to a user, are injected into the stack when run as the client Subject, to authenticate the user for the server and establish a secure connection, calls made by the client are run with the user's Subject on the server, again for access control purposes. This functionality is beyond the capability of Java RMI, we aren't using Java RMI to do this. This is very important to allow us to make fine grained access control decisions, or perform event notification callbacks over secure connections, without this feature, we can't make a secure connection with a callback, and you know what happens when you have to do something, but cannot do it securely? We only grant network access directly back to the server, downloaded code has been verified and is not expected to cause denial of service, by consuming resources etc, but we don't want to grant third party access to files, or random network connections, we still have privacy obligations for third party information. We can allow a third party to use unsigned certificates to sign their jar files or use a checksum and we verify them using a secure connection to the server, prior to loading. We then dynamically grant permissions to the server's self signed Certificate (used to sign the jar file), or a ProtectionDomain, after authenticating the server and receiving a check sum or certificate from it. So the client authenticates the server using signed TLS certificates (EG by letsencrypt.org or a trusted CA). We use self signed certificates on Jar files if we sign them, we are actually trusting the server entity in this case, eg a trusted company, but also placing restrictions on them. I am probably missing something, but I don't understand how this is secure if you are using TLS server certificates as the basis for authenticating signed code. These are two very different use cases. Clarifying the level of trust: 1. You have a trusted party, whom you trust to write their own code. 2. You run that code on your systems dynamically. 3. You trust the other party, but you either haven't or it's not practical to audit their code. 4. Using the principle of least privilege, you limit the ability of the other party's code to ensure they are unable observe data they shouldn't, eg a third party with whom you also do business. 5. While a trusted party (eg a supplier) could write code that caused denial of service, eg using up all available memory, there is no motivation for them to do so. However there may be motivation for them to see quotes on your system from another supplier if they have access to it. The code is loaded dynamically, but before the jvm loads it, we authenticate the party that is asking us to load their code, after which we are basically asking them the question, is this the code you want us to load? Please check that it hasn't been tampered with. The trusted party gives us a checksum, or a self signed certificate they used to sign the jar, we are then satisfied that we have received the software unaltered from the trusted party and not a MITM attack, so we load it. However we limit the permission of this software using the principle of least privilege. They don't get file permissions, they are only allowed to connect to the server they used to authenticate with. If a third party uses the same jar file, they don't gain the permissions granted to other parties, as it will be loaded into a separate ClassLoader with the permissions granted to that party only. Just to clarify, these are software logic interactions. If we remove access control, third parties will be able to open local network connections and freely and use Java Serialization over unsecured connections, exposing us to an attacker who can use a gadget attack. Presently they cannot open a network connection, access files or do much of anything without Permission. All those protections will be removed with this JEP. from https://community.letsencrypt.org/t/do-you-support-code-signing/370 Code-signing certificates as they’re used today are part of systems that try to decide whether a software source is malicious or legitimate. I don’t th
Re: Java Security: JEP: 411: Deprecate the Security Manager for Removal - What about Serialization?
On 4/05/2021 5:12 am, Sean Mullan wrote: -bcc jdk-dev -cc security-dev On 4/30/21 10:04 PM, Peter Firmstone wrote: In our software we use a ProtectionDomain to represent a remote server, because a thread only runs with the user's Subject (and that Subject must be carefully preserved for other threads), there is no way to represent the remote Server's Subject in a local domain , other than with a ProtectionDomain. Our software is peer to peer, clients can be servers and servers can also be clients. Code to interact with the server is downloaded via Maven and loaded. Any permission's granted to a user, are injected into the stack when run as the client Subject, to authenticate the user for the server and establish a secure connection, calls made by the client are run with the user's Subject on the server, again for access control purposes. This functionality is beyond the capability of Java RMI, we aren't using Java RMI to do this. This is very important to allow us to make fine grained access control decisions, or perform event notification callbacks over secure connections, without this feature, we can't make a secure connection with a callback, and you know what happens when you have to do something, but cannot do it securely? We only grant network access directly back to the server, downloaded code has been verified and is not expected to cause denial of service, by consuming resources etc, but we don't want to grant third party access to files, or random network connections, we still have privacy obligations for third party information. We can allow a third party to use unsigned certificates to sign their jar files or use a checksum and we verify them using a secure connection to the server, prior to loading. We then dynamically grant permissions to the server's self signed Certificate (used to sign the jar file), or a ProtectionDomain, after authenticating the server and receiving a check sum or certificate from it. So the client authenticates the server using signed TLS certificates (EG by letsencrypt.org or a trusted CA). We use self signed certificates on Jar files if we sign them, we are actually trusting the server entity in this case, eg a trusted company, but also placing restrictions on them. I am probably missing something, but I don't understand how this is secure if you are using TLS server certificates as the basis for authenticating signed code. These are two very different use cases. Clarifying the level of trust: 1. You have a trusted party, whom you trust to write their own code. 2. You run that code on your systems dynamically. 3. You trust the other party, but you either haven't or it's not practical to audit their code. 4. Using the principle of least privilege, you limit the ability of the other party's code to ensure they are unable observe data they shouldn't, eg a third party with whom you also do business. 5. While a trusted party (eg a supplier) could write code that caused denial of service, eg using up all available memory, there is no motivation for them to do so. However there may be motivation for them to see quotes on your system from another supplier if they have access to it. The code is loaded dynamically, but before the jvm loads it, we authenticate the party that is asking us to load their code, after which we are basically asking them the question, is this the code you want us to load? Please check that it hasn't been tampered with. The trusted party gives us a checksum, or a self signed certificate they used to sign the jar, we are then satisfied that we have received the software unaltered from the trusted party and not a MITM attack, so we load it. However we limit the permission of this software using the principle of least privilege. They don't get file permissions, they are only allowed to connect to the server they used to authenticate with. If a third party uses the same jar file, they don't gain the permissions granted to other parties, as it will be loaded into a separate ClassLoader with the permissions granted to that party only. If we remove access control, third parties will be able to open local network connections and freely and use Java Serialization over unsecured connections, exposing us to an attacker who can use a gadget attack. Presently they cannot open a network connection, access files or do much of anything without Permission. All those protections will be removed with this JEP. from https://community.letsencrypt.org/t/do-you-support-code-signing/370 Code-signing certificates as they’re used today are part of systems that try to decide whether a software source is malicious or legitimate. I don’t think Let’s Encrypt could easily play that kind of role when issuing certificates free of charge with an automated process without checking the real-wo
Re: New candidate JEP: 411: Deprecate the Security Manager for Removal
Thanks David, will make a note of it for future reference.
Cheers,
Peter.
On 30/04/2021 12:57 am, David Lloyd wrote:
If it helps, we've solved this particular problem in a couple of
places by using an MR-JAR which selects an implementation using
`StackWalker` when Java 9+ is used. I will say however that it
appears to be slightly less performant, which is unfortunate (but
hopefully fixable at some point in the future).
On Thu, Apr 29, 2021 at 1:48 AM Peter Firmstone
mailto:peter.firmst...@zeus.net.au>> wrote:
We also use the SecurityManager for caller sensitive method calls.
I re-implemented a secure implementation of Java Serialization,
using a
public API and fewer features (eg no circular links), in this
implementation, each class in an object's hierarchy has its own
namespace, the calling class is discovered using a SecurityManager
subclass.
--
Regards,
Peter Firmstone
0498 286 363
Zeus Project Services Pty Ltd.
On 29/04/2021 3:37 pm, Peter Firmstone wrote:
> Which version of Java is this planned for? Will the last version
> supporting the security manager be a long term support version, eg
> back ports of security patches and TLS technologies?
>
> We have our own security manager implementation and policy provider
> implementations. Both of these are high performance and
non-blocking
> and we are able to dynamically grant and revoke some permissions.
> While I acknowledge the Java policy implementation has a
significant
> performance impact, due to blocking permission checks, ours is less
> than 1%. Our software doesn't share PermissionCollection instances
> among threads, or even have a Permissions cache,
> PermissionCollection's are generated for each permission check and
> discarded for garbage collection, the Permission object
themselves are
> cached (after initialization and safe publication), as are the
results
> of repeated permission checks. We also have our own Permission
> implementations.
>
> We have tools that generate policy files with least privilege,
> although we will manually alter them with wildcards, for network
> connections for instance.
>
> In our software, dynamic permissions are granted after
authentication
> of TLS connections.
>
> It is too early for me to tell if there are suitable replacement
> technologies available. I can understand the motivation for
reducing
> Java's software development burden, but I think this version of
Java
> might be the last for us, it would certainly be good if a long term
> support version was available, perhaps indefinitely lol.
>
> Regards,
>
> Peter Firmstone
> Zeus Project Services Pty Ltd.
>
>
> On 16/04/2021 4:05 am, mark.reinh...@oracle.com
<mailto:mark.reinh...@oracle.com> wrote:
>> https://openjdk.java.net/jeps/411
<https://openjdk.java.net/jeps/411>
>>
>> Summary: Deprecate the Security Manager for removal in a future
>> release. The Security Manager dates from Java 1.0. It has
not been
>> the
>> primary means of securing client-side Java code for many years,
>> and it
>> has rarely been used to secure server-side code. To move Java
>> forward,
>> we intend to deprecate the Security Manager for removal in
concert
>> with
>> the legacy Applet API (JEP 398).
>>
>> - Mark
>
--
- DML • he/him
Re: New candidate JEP: 411: Deprecate the Security Manager for Removal
On 29/04/2021 10:57 pm, Sean Mullan wrote: On 4/29/21 1:37 AM, Peter Firmstone wrote: We have our own security manager implementation and policy provider implementations. Both of these are high performance and non-blocking and we are able to dynamically grant and revoke some permissions. While I acknowledge the Java policy implementation has a significant performance impact, due to blocking permission checks, ours is less than 1%. Our software doesn't share PermissionCollection instances among threads, or even have a Permissions cache, PermissionCollection's are generated for each permission check and discarded for garbage collection, the Permission object themselves are cached (after initialization and safe publication), as are the results of repeated permission checks. We also have our own Permission implementations. Yes, you and I had discussions about this several years ago and I believe I asked if you would be willing to donate these performance improvements. I can't remember but I think you had interest in doing that but couldn't make it happen for some reason. Licensing, parts of the code I could donate as I am the original author, but other parts were AL 2.0 licensed by others, I think I found some AL2.0 code in OpenJDK at the time, but I was having trouble getting clarity around licensing, I don't remember the details but I think I needed to contact the other authors and have them sign contributor agreements or rewrite their code. The two components that caused licensing issues were the policy parser (although Java has it's own, so not a major issue) and an RFC 3986 URI implementation which replaced URL, which was heavily utilised, it normalizes URI for policy implies checks, including RFC 5952 IPv6 address normalization, OS specific file based URI were normalized dependent on the underlying OS (eg uppercase for Microsoft). Using Java's URI wasn't an option as it wouldn't normalize appropriately, while using URL would cause blocking network calls. It is a strict RFC 3986 implementation, Unicode characters beyond US-ASCII are escaped, and it utilises bit shift operations during normalization, it's also immutable and not serializable, so it wasn't a backward compatible replacement for Java's URI. The difference between URI and URL use in policy files, was that if two different domain names resolved to the same IP Address, they implied each other under a URL ( required blocking network dns calls), while they didn't imply one another under URI. This had the added benefit of allowing one domain name to resolve to multiple IP addresses without having to add multiple IP addresses to policy files, for example fail-over web server addresses. Some trivia, I also wrote a URIClassLoader that replaced URLClassLoader, specifically to override behaviour in SecureClassLoader, where URL's are used as key's in a cache, again for performance reasons. Re-implementing RFC 3986 URI functionality was a significant undertaking, I wasn't familiar with OpenJDK's build system, was busy with other things so I put it off until later... which hasn't happened yet lol. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: JEP411: Missing use-case: Monitoring / restricting libraries
Having implemented SecurityManager and Policy providers, I'd like to
comment on some of the assessments, some thoughts:
* Poor performance, this is specific to the Java Policy
implementation, I have addressed this in my implementations,
performance impact is imperceptible, I know how to address it.
* Brittle permission model - Agreed, it is brittle.
o Developers should use doPrivileged blocks to make security
sensitive calls, so only one domain is on the call stack. Agreed
when they don't do this it causes viral permission expansion.
o Implementing Deny would address the example given.
o Un-trusted code should never be run on the JVM, however some
code is more trusted, or well audited, so we may wish to limit
code that doesn't require privileges, based on principle of
least privilege.
o Is there a way to simplify Permission's? We can write tools to
generate policy files (I have).
* Difficult programming model
o If the library developer uses doPrivileged blocks and documents
the permissions, then the application code doesn't require those
permissions, as it is not on the call stack.
o Agreed that most developers don't do this.
o I use a tool to generate policy files that capture the viral
growth of permissions, these still require some editing, eg to
add system properties.
Is there a simpler way to limit permissions of library code?
I have gotten so used to secure coding, because we use it, I don't find
it difficult, writing non blocking or good concurrent code is more
difficult and developers make lots of mistakes with race conditions,
visibility and synchronization.
There are good static code analysis tools that identify poor coding
practices, the same tools could be updated to include secure coding
practices as well, to address viral permission expansion.
Perhaps if we instead address the performance and usability issues, we
could improve adoption,so it adds to Java's appeal, rather than
detracting from it?
Regards,
Peter.
Comments and code from one of my policy implementations (I have a few
policy implementations that use the decorator pattern to add
functionality, like dynamic permission grants):
* If there is sufficient interest, we can implement a DENY clause,
* in this case DENY cannot apply to GRANT clauses that contain
* {@link java.security.AllPermission}, the domains to which a DENY clause
* would apply will be a less privileged domain. For example a user
could be
* granted SocketPermission("*", "connect"), while a DENY clause might
* list specific SocketPermission domains that are disallowed, where a
DENY
* clause has precedence over all GRANT clause Permissions except for
AllPermission.
@Override
public boolean implies(ProtectionDomain domain, Permission
permission) {
if (permission == null) throw new
NullPointerException("permission not allowed to be null");
if (domain == myDomain) {
PermissionCollection pc = myPermissions;
return pc.implies(permission);
}
Class klass = permission.getClass();
// Need to have a list of Permission's we can sort if
permission is SocketPermission.
NavigableSet perms = new
TreeSet(comparator);
PermissionGrant [] grantRefCopy = grantArray;
int l = grantRefCopy.length;
/* Check for privileged grants first to avoid recursion when
* privileged domains become involved in policy decisions */
for (int j = 0; j < l; j++){
PermissionGrant ge = grantRefCopy[j];
if (ge.isPrivileged()){
if (ge.implies(domain)){
return true;
}
}
}
/* Merge the static Permissions, check for Privileged */
PermissionCollection staticPC = null;
if (domain != null) {
staticPC =domain.getPermissions();
if (staticPC != null){
Enumeration e = staticPC.elements();
while (e.hasMoreElements()){
Permission p = e.nextElement();
// return early if possible.
if (p instanceof AllPermission ) return true;
// Only add relevant permissions to minimise size.
if (klass.isInstance(permission) || permission
instanceof UnresolvedPermission){
perms.add(p);
}
}
}
}
/* Check less privileged grants */
for ( int j =0; j < l; j++ ){
PermissionGrant ge = grantRefCopy[j];
if (!ge.isPrivileged()){
if (ge.implies(domain)){
Collection c = ge.getPermissions();
Iterator i = c.iterator();
while (i.hasNext()){
Permission p = i.next();
// Only add re
Re: New candidate JEP: 411: Deprecate the Security Manager for Removal
We also use the SecurityManager for caller sensitive method calls. I re-implemented a secure implementation of Java Serialization, using a public API and fewer features (eg no circular links), in this implementation, each class in an object's hierarchy has its own namespace, the calling class is discovered using a SecurityManager subclass. -- Regards, Peter Firmstone 0498 286 363 Zeus Project Services Pty Ltd. On 29/04/2021 3:37 pm, Peter Firmstone wrote: Which version of Java is this planned for? Will the last version supporting the security manager be a long term support version, eg back ports of security patches and TLS technologies? We have our own security manager implementation and policy provider implementations. Both of these are high performance and non-blocking and we are able to dynamically grant and revoke some permissions. While I acknowledge the Java policy implementation has a significant performance impact, due to blocking permission checks, ours is less than 1%. Our software doesn't share PermissionCollection instances among threads, or even have a Permissions cache, PermissionCollection's are generated for each permission check and discarded for garbage collection, the Permission object themselves are cached (after initialization and safe publication), as are the results of repeated permission checks. We also have our own Permission implementations. We have tools that generate policy files with least privilege, although we will manually alter them with wildcards, for network connections for instance. In our software, dynamic permissions are granted after authentication of TLS connections. It is too early for me to tell if there are suitable replacement technologies available. I can understand the motivation for reducing Java's software development burden, but I think this version of Java might be the last for us, it would certainly be good if a long term support version was available, perhaps indefinitely lol. Regards, Peter Firmstone Zeus Project Services Pty Ltd. On 16/04/2021 4:05 am, mark.reinh...@oracle.com wrote: https://openjdk.java.net/jeps/411 Summary: Deprecate the Security Manager for removal in a future release. The Security Manager dates from Java 1.0. It has not been the primary means of securing client-side Java code for many years, and it has rarely been used to secure server-side code. To move Java forward, we intend to deprecate the Security Manager for removal in concert with the legacy Applet API (JEP 398). - Mark
Re: New candidate JEP: 411: Deprecate the Security Manager for Removal
Which version of Java is this planned for? Will the last version supporting the security manager be a long term support version, eg back ports of security patches and TLS technologies? We have our own security manager implementation and policy provider implementations. Both of these are high performance and non-blocking and we are able to dynamically grant and revoke some permissions. While I acknowledge the Java policy implementation has a significant performance impact, due to blocking permission checks, ours is less than 1%. Our software doesn't share PermissionCollection instances among threads, or even have a Permissions cache, PermissionCollection's are generated for each permission check and discarded for garbage collection, the Permission object themselves are cached (after initialization and safe publication), as are the results of repeated permission checks. We also have our own Permission implementations. We have tools that generate policy files with least privilege, although we will manually alter them with wildcards, for network connections for instance. In our software, dynamic permissions are granted after authentication of TLS connections. It is too early for me to tell if there are suitable replacement technologies available. I can understand the motivation for reducing Java's software development burden, but I think this version of Java might be the last for us, it would certainly be good if a long term support version was available, perhaps indefinitely lol. Regards, Peter Firmstone Zeus Project Services Pty Ltd. On 16/04/2021 4:05 am, mark.reinh...@oracle.com wrote: https://openjdk.java.net/jeps/411 Summary: Deprecate the Security Manager for removal in a future release. The Security Manager dates from Java 1.0. It has not been the primary means of securing client-side Java code for many years, and it has rarely been used to secure server-side code. To move Java forward, we intend to deprecate the Security Manager for removal in concert with the legacy Applet API (JEP 398). - Mark
Re: JDK 14 RFR of JDK-8231262: Suppress warnings on non-serializable instance fields in security libs serializable classes
Hello, I'd make an exception for interfaces, often these are not serializable, but their implementations may be, in this case a warning would be spurious. Regards, Peter. On 20/09/2019 3:32 AM, Joe Darcy wrote: Hello, Ahead of augmenting javac's serial lint checks under JDK-8160675, it would be helpful to mark fields in security libs classes where the class is serializable, but a non-transient instance field does *not* have a serialiable type. Such classes may have difficulties being serialized at runtime: JDK-8231262 : Suppress warnings on non-serializable instance fields in security libs serializable classes http://cr.openjdk.java.net/~darcy/8231262.0/ The review thread of the of the analogous core libs change, JDK-8231202: "Suppress warnings on non-serializable non-transient instance fields in serializable classes", is out on core-libs: http://mail.openjdk.java.net/pipermail/core-libs-dev/2019-September/062456.html Thanks, -Joe
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Hi Alan, Your suspicion is correct. :) Thanks for the leads, I'll look into it further. Currently the policy implementation finds policy url's in system properties, "java.security.policy" and numbered policy locations with the prefix "policy.url." if the "java.security.policy" property doesn't begin with "=" (which represents java.security.policy==). Cheers, Peter. On 15/09/2019 10:58 PM, Alan Bateman wrote: On 14/09/2019 21:21, Peter Firmstone wrote: Hi Alan, We've got a bunch of very old policy files in our test suite, so they still had policy grants using the extension directory property. The grant for the extension directory property was followed by a forward slash and asterix. Oddly when the property was missing the grant became a wildcard URL. Note this isn't the sun PolicyFile implementation, but our policy provider also augments, rather than replace, maybe there's a new policy file our provider isn't aware of? From memory there was something special about the way the extension directory property was treated by the policy provider, but I don't recall the details, the same problems don't appear to exist when other properties in policy files cannot be resolved. Modules that required permissions, seem to be service providers: In jdk/jdk repo, the following policy files are merged in the build to create the default policy: src/java.base/windows/lib/security/default.policy src/java.base/solaris/lib/security/default.policy src/java.base/share/lib/security/default.policy The default policy goes into a JDK internal location in the run-time image and used by the PolicyFile implementation. If you look in there you should see the permissions that are granted to the modules that map to the platform class loader. The intention is that deployments that are setting their own policy files don't need to be concerned about the permissions of modules in the run-time image. I suspect you are looking for a custom PolicyFile implementation to make use of these defaults to avoid needing to be concerned with the specific permissions that the modules in the run-time image. -Alan
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Hi Alan,
We've got a bunch of very old policy files in our test suite, so they
still had policy grants using the extension directory property. The
grant for the extension directory property was followed by a forward
slash and asterix. Oddly when the property was missing the grant became
a wildcard URL. Note this isn't the sun PolicyFile implementation, but
our policy provider also augments, rather than replace, maybe there's a
new policy file our provider isn't aware of?
From memory there was something special about the way the extension
directory property was treated by the policy provider, but I don't
recall the details, the same problems don't appear to exist when other
properties in policy files cannot be resolved.
Modules that required permissions, seem to be service providers:
grant codebase "jrt:/jdk.security.auth" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/jdk.crypto.cryptoki" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/java.smartcardio" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/java.xml.crypto" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/java.security.jgss" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/jdk.crypto.ec" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/jdk.crypto.mscapi" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/jdk.localedata" {
permission java.security.AllPermission "", "";
};
grant codebase "jrt:/jdk.security.jgss" {
permission java.security.AllPermission "", "";
};
Regards,
Peter.
On 14/09/2019 5:22 PM, Alan Bateman wrote:
On 13/09/2019 23:07, Peter Firmstone wrote:
:
One change I noticed is permissions granted to the java extension
directory are now granted to every domain in our policy provider as
the java.ext.dirs property is now blank, I also had to grant
permissions to a number of jdk modules, after fixing these, everthing
running as expected, except for a few minor test failures.
The extension mechanism was deprecated in one of the maintenance
releases of the JSR for Java SE 8 so hopefully not a surprise that the
system property java.ext.dirs is no longer set. Since JDK 9, the VM
will will not start if this system property is set on the command line
and there is also a XX option in JDK 8 to help find usages of this
mechanism. So I think I'm surprised that permissions are being granted
when the property isn't set.
Also surprised to hear that you need to grant permissions to the
java.* or jdk.* modules. Are you saying that that the permissions
granted in the default JDK policy file are incomplete or that there
are code is missing a call to doPrivileged somewhere? Part of the
reason for asking is that the both "-Djava.security.policy=" and
"-Djava.security.policy==" will augment rather than override the
default permissions. Maybe you are doing something that doesn't use
this mechanism and you are always overriding the permissions granted
to the modules in the run-time image?
-Alan
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Hi Claes,
So this security manager is part of a much larger program, (a fork of
Jini / Apache River), I've almost finished the transition from Java 8 to
Java 11...
One change I noticed is permissions granted to the java extension
directory are now granted to every domain in our policy provider as the
java.ext.dirs property is now blank, I also had to grant permissions to
a number of jdk modules, after fixing these, everthing running as
expected, except for a few minor test failures.
Next step will be to test against the EA builds.
On 17/08/2019 7:24 AM, Peter Firmstone wrote:
Thanks Claes,
I'll run some tests :)
Cheers,
Peter.
On 16/08/2019 9:14 PM, Claes Redestad wrote:
Hi Peter,
by explicitly ensuring the file system has been initialized before
installing a SecurityManager using a hook in System.setSecurityManager,
the patch at hand takes step to ensure things stay neutral w.r.t.
Permission initialization order when using any SecurityManager. It's not
perfectly identical, so while unlikely, there's a theoretical
possibility some implementation scenario not covered by our regression
tests might run into issues. Any help testing custom implementation on
coming EA builds would be greatly appreciated!
One thing we could do on the JDK end to reduce fragility somewhat in
this area is to provoke initialization of
sun.security.util.SecurityConstants before installing the first SM.
/Claes
On 2019-08-16 12:40, Peter Firmstone wrote:
Hello Alan,
Yes, we are aware of those issues.
I mean documenting that system Permission classes should be loaded
before setting a custom SecurityManager, accessing the file system
is important, so if you haven't loaded the necessary classes before
setting a custom SecurityManager, it won't be gracefull... The
simplest way of ensuring classes are loaded is by creating object
instances of them.
Perhaps just a note to beware of ensuring necessary classes are
loaded and let developers figure out what they need.
The recursive calls are easy enough to deal with to avoid any stack
overflows using ThreadLocal.
inTrustedCodeRecursiveCall.set(Boolean.TRUE);
try {
delegateContext = AccessController.doPrivileged(
new PrivilegedAction(){
public AccessControlContext run() {
return new
AccessControlContext(finalExecutionContext, dc);
}
}
);
}finally {
inTrustedCodeRecursiveCall.set(Boolean.FALSE); //
Must always happen, no matter what.
}
We've only really been caught out once with some jvm bootstrap
changes, otherwise it's been rock solid.
The other thing we do is once we've got more than three
ProtectionDomains on the stack is execute the ProtectionDomain
implies checks in parallel. Really helps when you're making a lot
of network calls.
Cheers,
Peter.
On 16/08/2019 5:04 PM, Alan Bateman wrote:
On 15/08/2019 23:20, Peter Firmstone wrote:
:
The following code is included in the constructor of our
SecurityManager implementation, I suspect we may need to add some
classes to this list, perhaps this is something that needs
documenting?
The checkPermission method of custom security manager can run
arbitrary code so recursive initialization, stack overflow,
bootstrap method errors, ... are always hazards. I don't know what
documentation you have in mind but I don't think there is a
definite list of classes that need to be loaded/initialized before
the custom security manager is set because it will come down to the
code that it executes in its checkPermission method.
-Alan.
Re: Serialzation PREVIOUSLY: RFR: 8229773: Resolve permissions for code source URLs lazily
I probably should have vetted this before hitting send... let me know if
you need any clarifications.
Cheers,
Peter.
On 23/08/2019 12:59 PM, Peter Firmstone wrote:
"...since at the time the industry believed that distributed objects
were going to save us from complexity.) Many of the sins of
serialization were committed in the desire to get that last .1%, but
the cost and benefit of that last .1% are woefully out of balance."
The following are probably a non goals, but something to consider or
keep in mind, relating to distributed objects:
The are three types of distributed objects:
1. Immutable value / data Object types.
2. Shared Mutable Objects.
3. Unshared Mutable Objects.
4. Remote Objects / Services (best for managing shared mutable state).
The second type of distributed object causes much pain and should be
discouraged. The first three types of distributed objects can have
class resolution issues, but these are solveable.
A lot of folks also have problems with deserialization Objects when
class visibility is different at both ends, I'm guessing this would be
the same for value types.
For example OSGi folk recommend using primitive parameter types for
remote OSGi services.
RMI annotates streams with codebase annotations. Jini Extensible
Remote Invocation used to do that too.
The problem with RMI codebase and Jini codebase annotations is if you
resolve your classes locally, you lose the codebase annotations, when
re-serializing data and because class visibility can be different at
different endpoints, end up with all sorts of class resolution
issues. "Class Loading Issues in Java™ RMIand Jini™ Network
Technology" by Michael Warres
https://pdfs.semanticscholar.org/143f/468fcbdafd20f2b8c27fe5e0a869913b641a.pdf
The solution of course is simple, ensure that you deserialize into the
same module that you serialized from, especially when deserializing in
another jvm, so class resolution is identical.
We serialize a lot of complex object graphs, none are circular. The
module used for serialization should have visiblity of the entire
graph of object classes.
So if we're using OSGi modules, and provide a network / remote service
(not to be confused with an OSGi remote service) we ensure the proxy's
for these services have the same module installed at the client and
server endpoints. The service is represented by a Java interface and
the client makes calls on the interfaces methods. This interface may
be implemented by what is called a smart proxy, which is encapsulated
by a module which is dynamically downloaded at runtime, or a
reflection Proxy using an InvocationHandler that is generated
dynamically.
We still provide an option for codebase annotations for client
parameter objects, where a client subclasses parameter types and pass
them to the service, but this is discouraged, it is provided for
backward compatibility only. Where the parameters are also
interfaces, the client can implement a remote object and pass it as a
parameter instead, in our system, this will cause a module to be
loaded in the server identical to that at the client to resolve the
remote object classes, without using stream codebase annotations.
Incidentally, if you're curious how this happens, a proxy is sent {I
guess you can call it a serialization proxy :) } and authenticated by
the remote end, security constraints applied, then the remote end asks
the proxy for a codebase URL,which is loaded into a ClassLoader with
controlled visibility, this is extensible using a ServiceProvider or
OSGi service, then the proxy is deserialized into this by calling a
method on the serialization proxy.
By limiting scope, we can still have 99% of the benefits of
distributed objects, without the pain.
Incidentally apart from the complexity of class resolution, what
really limited distributed computing was IPv4. IPv6 removes the
network addressing limitations placed on distributed computing.
So I'd make the following qualifications:
1. Use only primitive types when serializing between different
languages.
2. Serialize Java language Object types and primitives only between
jvm's when class visibility is uncontrolled.
3. When serializing other object types, ensure they are immutable if
shared and that class visibility is identical and managed at both
endpoints.
4. Do not serialize objects whose classes may not be resolveable
(when you need to depend on annotated streams and uncontrolled
class resolution for example), find another way to solve the
problem.
We've had a 20 years to iron out the wrinkles. :)
Regards,
Peter.
On 23/08/2019 7:36 AM, Peter Firmstone wrote:
Hi Sean,
Regarding the section entitled "Why not write a new serialization
library?", unlike the serialization libraries listed, our purpose was
to be able to securely deserialize untrusted
Re: Serialzation PREVIOUSLY: RFR: 8229773: Resolve permissions for code source URLs lazily
"...since at the time the industry believed that distributed objects
were going to save us from complexity.) Many of the sins of
serialization were committed in the desire to get that last .1%, but the
cost and benefit of that last .1% are woefully out of balance."
The following are probably a non goals, but something to consider or
keep in mind, relating to distributed objects:
The are three types of distributed objects:
1. Immutable value / data Object types.
2. Shared Mutable Objects.
3. Unshared Mutable Objects.
4. Remote Objects / Services (best for managing shared mutable state).
The second type of distributed object causes much pain and should be
discouraged. The first three types of distributed objects can have
class resolution issues, but these are solveable.
A lot of folks also have problems with deserialization Objects when
class visibility is different at both ends, I'm guessing this would be
the same for value types.
For example OSGi folk recommend using primitive parameter types for
remote OSGi services.
RMI annotates streams with codebase annotations. Jini Extensible Remote
Invocation used to do that too.
The problem with RMI codebase and Jini codebase annotations is if you
resolve your classes locally, you lose the codebase annotations, when
re-serializing data and because class visibility can be different at
different endpoints, end up with all sorts of class resolution issues.
"Class Loading Issues in Java™ RMIand Jini™ Network Technology" by
Michael Warres
https://pdfs.semanticscholar.org/143f/468fcbdafd20f2b8c27fe5e0a869913b641a.pdf
The solution of course is simple, ensure that you deserialize into the
same module that you serialized from, especially when deserializing in
another jvm, so class resolution is identical.
We serialize a lot of complex object graphs, none are circular. The
module used for serialization should have visiblity of the entire graph
of object classes.
So if we're using OSGi modules, and provide a network / remote service
(not to be confused with an OSGi remote service) we ensure the proxy's
for these services have the same module installed at the client and
server endpoints. The service is represented by a Java interface and
the client makes calls on the interfaces methods. This interface may
be implemented by what is called a smart proxy, which is encapsulated by
a module which is dynamically downloaded at runtime, or a reflection
Proxy using an InvocationHandler that is generated dynamically.
We still provide an option for codebase annotations for client parameter
objects, where a client subclasses parameter types and pass them to the
service, but this is discouraged, it is provided for backward
compatibility only. Where the parameters are also interfaces, the
client can implement a remote object and pass it as a parameter instead,
in our system, this will cause a module to be loaded in the server
identical to that at the client to resolve the remote object classes,
without using stream codebase annotations.
Incidentally, if you're curious how this happens, a proxy is sent {I
guess you can call it a serialization proxy :) } and authenticated by
the remote end, security constraints applied, then the remote end asks
the proxy for a codebase URL,which is loaded into a ClassLoader with
controlled visibility, this is extensible using a ServiceProvider or
OSGi service, then the proxy is deserialized into this by calling a
method on the serialization proxy.
By limiting scope, we can still have 99% of the benefits of distributed
objects, without the pain.
Incidentally apart from the complexity of class resolution, what really
limited distributed computing was IPv4. IPv6 removes the network
addressing limitations placed on distributed computing.
So I'd make the following qualifications:
1. Use only primitive types when serializing between different languages.
2. Serialize Java language Object types and primitives only between
jvm's when class visibility is uncontrolled.
3. When serializing other object types, ensure they are immutable if
shared and that class visibility is identical and managed at both
endpoints.
4. Do not serialize objects whose classes may not be resolveable
(when you need to depend on annotated streams and uncontrolled
class resolution for example), find another way to solve the problem.
We've had a 20 years to iron out the wrinkles. :)
Regards,
Peter.
On 23/08/2019 7:36 AM, Peter Firmstone wrote:
Hi Sean,
Regarding the section entitled "Why not write a new serialization
library?", unlike the serialization libraries listed, our purpose was
to be able to securely deserialize untrusted data, while maintaining
backward serial form compatibility with Java Serialization, provided
it didn't compromise security.
We don't use blacklists or whitelis
Re: Serialzation PREVIOUSLY: RFR: 8229773: Resolve permissions for code source URLs lazily
Hi Sean, Regarding the section entitled "Why not write a new serialization library?", unlike the serialization libraries listed, our purpose was to be able to securely deserialize untrusted data, while maintaining backward serial form compatibility with Java Serialization, provided it didn't compromise security. We don't use blacklists or whitelists, we use permissions to grant DeserializationPermission, it doesn't have the granularity of white lists, but then, classes that implement @AtomicSerial are supposed to be hardened implementations in any case. If it can be of use, feel free to experiment with it, hopefully it might help with some of your design decisions: https://github.com/pfirmstone/JGDMS/tree/trunk/JGDMS/jgdms-platform/src/main/java/org/apache/river/api/io Much of the code on this site provides implementation examples as well. Regards, Peter. On 20/08/2019 7:55 AM, Sean Mullan wrote: Brian Goetz (copied) has done a lot of thinking in the serialization area, so I have copied him. Not sure if you have seen it but he recently posted a document about some of his ideas and possible future directions for serialization: http://cr.openjdk.java.net/~briangoetz/amber/serialization.html --Sean On 8/17/19 10:22 PM, Peter Firmstone wrote: Thanks Sean, You've gone to some trouble to answer my question, which demonstrates you have considered it. I donate some time to help maintain Apache River, derived from Sun's Jini. Once Jini depended on RMI, today, not so much, it still has some dependencies on some RMI interfaces, but doesn't utilise JRMP although it provides some backward compatibilty enable it. But my point is, we heavily utilise java Serialization, and have an independant implementation of a subset of Java Serialization (originating from Apache Harmony). We do this for security as we use an annotated serialization constructor. Serial form is unchanged, we have Serializers for commonly used java library objects, for example, we have a "PermissionSerializer", but we don't have a "PermissionCollectionSerializer" or "PermissionsSerializer" (for java.security.Permissions). Incidentally, we have found we do not need the ability to serialize circular object graphs. Throwable is an object that has a circular object graph, but that circular object graph can be linked up after deserialization. Permission implementing Serializable is probably not too much of a threat, as these objects are effectively immutable after lazy initialization. ProtectionDomain calls java.security.Permissions::setReadOnly during it's construction. ProtectionDomain::getPermissions returns internal java.security.Permissions. If this is serialized, then the readOnly internal state can be written to as the internal object references are accessible from within the stream. Admitedly, the attacker would already need to have some privilege, to have access to a ProtectionDomain, so it's a path of privilege escallation. I'm not talking about gadget attacks and deserialization of untrusted data, I'm talking about breaking encapsulation. Even though we are heavily dependant on Java Serialization, we are very careful when we implement it, and avoid implementing it when possible. Hindsight is 20:20, but given we are now seeing some Java SE backward compatibility breakages, perhaps it might be worth considering breaking serialization. I don't mean we need to necessarily break object serial form, but making the Java serialization API explicit with subset of existing api features, that makes long term maintenace and security less of a burden and removing support for Serialization of some objects, where it is seldom used, perhaps using a JEP that requests developers to consider which library objects actually need to be serializable. Something we do in our Java Serialization API is require that mutable deserialized objects are defensively copied during object construction (serial fields are deserialized before an object is constructed, the deserialized fields are accessible via a parameter passed in during construction. We have tools that assist developers to check deserialized Java Collections contain the expected object types for example, so during object construction the developer has to replace the Collection with a new instance and copy the contents to the new Collection after checking the type of each object contained therein. Also we don't actually serialize Java Collections, we have standard serial forms for List, Set and Map, so these serial forms are equal, similar to the List, Set and Map contracts. By doing this, Collections don't actually need to implement Serializable at all, as a Serializer becomes responsible for their serialization. This also means that all Collections must be accessed by interfaces, rather tha
Re: Serialzation PREVIOUSLY: RFR: 8229773: Resolve permissions for code source URLs lazily
Thanks Sean, No I hadn't seen it, I've just read it, will probably need to read it again to appreciate it fully... It certainly identifies all the issues I'm aware of, as well as being respectful of the original implementors (many of whom participated in Apache River when Jini was donated to Apache), I came to the same conclusion with circular object graphs; the benefits don't outweigh the cost. We also use annotations instead of interfaces,to annotate the class and constructor, so that overriding classes don't automagically inherit the functionality. At this time, we haven't reimplemented deconstruction, we are using ObjectOutputStream with serializers, which are basically serialization proxy's for existing classes, we have fully reimplemented deserialization using constructors. Agree with serial from being independant of the wire protocol, so any serialization scheme can be used, this is an excellent idea of course. The constructors / deconstructors have identified that serial form is really just a parameter list. Developers will want to make defensive copies of mutable state, just like public api methods. We did consider constructors with multiple parameters, but decided against it for the following reasons: 1. We didn't care about parameter order (tuples), or the order in which they were serialized / deserialized, we only cared about parameter names and types. 2. For encapsulation we didn't want subclasses having to manage the serial form of superclasses, we wanted them to remain as independant as possible, so they don't inadvertantly break. * For example, a library superclass adds a serial form parameter, or changes a type, in its serial form. The child class would have to be aware of the changes in order to pass the correct parameters to the correct superclass constructor. * Different serial version constructors would result in the loss of later version superclass state when child classes call an earlier version. 3. We settled on a caller sensitive parameter that is passed to the deserialization constructor. * Encapsulation: Each class in an inheritance heirarch only has access to it's own serial form. * The serial form of each class is independant and may evolve independantly. * Each class in the inheritance heirarchy is responsible for checking it's own invariants, including the ability to create superclass instances, even if a superclass is abstract for checking inter class invariants. 4. It was less work for the framework to populate a standard parameter object, with serial form, the framework didn't need to worry about inspecting the constructor signature and determining the parameter order. 5. One constructor could be used for different versions. 6. We currently use |serialPersistentFields to declare serial form, but there is probably a better way of doing this, perhaps a way that also documents different serial form versions.| Regards, Peter. On 20/08/2019 7:55 AM, Sean Mullan wrote: Brian Goetz (copied) has done a lot of thinking in the serialization area, so I have copied him. Not sure if you have seen it but he recently posted a document about some of his ideas and possible future directions for serialization: http://cr.openjdk.java.net/~briangoetz/amber/serialization.html --Sean On 8/17/19 10:22 PM, Peter Firmstone wrote: Thanks Sean, You've gone to some trouble to answer my question, which demonstrates you have considered it. I donate some time to help maintain Apache River, derived from Sun's Jini. Once Jini depended on RMI, today, not so much, it still has some dependencies on some RMI interfaces, but doesn't utilise JRMP although it provides some backward compatibilty enable it. But my point is, we heavily utilise java Serialization, and have an independant implementation of a subset of Java Serialization (originating from Apache Harmony). We do this for security as we use an annotated serialization constructor. Serial form is unchanged, we have Serializers for commonly used java library objects, for example, we have a "PermissionSerializer", but we don't have a "PermissionCollectionSerializer" or "PermissionsSerializer" (for java.security.Permissions). Incidentally, we have found we do not need the ability to serialize circular object graphs. Throwable is an object that has a circular object graph, but that circular object graph can be linked up after deserialization. Permission implementing Serializable is probably not too much of a threat, as these objects are effectively immutable after lazy initialization. ProtectionDomain calls java.security.Permissions::setReadO
Serialzation PREVIOUSLY: RFR: 8229773: Resolve permissions for code source URLs lazily
Thanks Sean, You've gone to some trouble to answer my question, which demonstrates you have considered it. I donate some time to help maintain Apache River, derived from Sun's Jini. Once Jini depended on RMI, today, not so much, it still has some dependencies on some RMI interfaces, but doesn't utilise JRMP although it provides some backward compatibilty enable it. But my point is, we heavily utilise java Serialization, and have an independant implementation of a subset of Java Serialization (originating from Apache Harmony). We do this for security as we use an annotated serialization constructor. Serial form is unchanged, we have Serializers for commonly used java library objects, for example, we have a "PermissionSerializer", but we don't have a "PermissionCollectionSerializer" or "PermissionsSerializer" (for java.security.Permissions). Incidentally, we have found we do not need the ability to serialize circular object graphs. Throwable is an object that has a circular object graph, but that circular object graph can be linked up after deserialization. Permission implementing Serializable is probably not too much of a threat, as these objects are effectively immutable after lazy initialization. ProtectionDomain calls java.security.Permissions::setReadOnly during it's construction. ProtectionDomain::getPermissions returns internal java.security.Permissions. If this is serialized, then the readOnly internal state can be written to as the internal object references are accessible from within the stream. Admitedly, the attacker would already need to have some privilege, to have access to a ProtectionDomain, so it's a path of privilege escallation. I'm not talking about gadget attacks and deserialization of untrusted data, I'm talking about breaking encapsulation. Even though we are heavily dependant on Java Serialization, we are very careful when we implement it, and avoid implementing it when possible. Hindsight is 20:20, but given we are now seeing some Java SE backward compatibility breakages, perhaps it might be worth considering breaking serialization. I don't mean we need to necessarily break object serial form, but making the Java serialization API explicit with subset of existing api features, that makes long term maintenace and security less of a burden and removing support for Serialization of some objects, where it is seldom used, perhaps using a JEP that requests developers to consider which library objects actually need to be serializable. Something we do in our Java Serialization API is require that mutable deserialized objects are defensively copied during object construction (serial fields are deserialized before an object is constructed, the deserialized fields are accessible via a parameter passed in during construction. We have tools that assist developers to check deserialized Java Collections contain the expected object types for example, so during object construction the developer has to replace the Collection with a new instance and copy the contents to the new Collection after checking the type of each object contained therein. Also we don't actually serialize Java Collections, we have standard serial forms for List, Set and Map, so these serial forms are equal, similar to the List, Set and Map contracts. By doing this, Collections don't actually need to implement Serializable at all, as a Serializer becomes responsible for their serialization. This also means that all Collections must be accessed by interfaces, rather than implementation classes, so the deserialization constructor, must defensively copy them into their preferred Collection instance. It's a bit like dependency injection. I know it would take time, and there would be some pain, but long term it would save a lot of maintenance developer time. Regards, Peter. On 17/08/2019 12:50 AM, Sean Mullan wrote: On 8/15/19 8:18 PM, Peter Firmstone wrote: Hi Roger, +1 for writeReplace Personally I'd like to see some security classes break backward compatibility and remove support for serialization as it allows someone to get references to internal objects, especially since these classes are cached by the JVM. Which makes PermissionCollection.setReadOnly() very easy to bypass, by adding permissions to internal collections once you have a reference to them. Does anyone have any use cases for serializing these objects? These objects are easy to re-create by sending or recieving and parsing strings, because they are built from text based policy files, and when you do that, you are validating input, so I never did fully understand why they were made serializable. This is briefly explained on page 61 in the "Inside Java 2 Platform Security" book [1]: "The Permission class implements two interfaces: java.security
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Thanks Claes,
I'll run some tests :)
Cheers,
Peter.
On 16/08/2019 9:14 PM, Claes Redestad wrote:
Hi Peter,
by explicitly ensuring the file system has been initialized before
installing a SecurityManager using a hook in System.setSecurityManager,
the patch at hand takes step to ensure things stay neutral w.r.t.
Permission initialization order when using any SecurityManager. It's not
perfectly identical, so while unlikely, there's a theoretical
possibility some implementation scenario not covered by our regression
tests might run into issues. Any help testing custom implementation on
coming EA builds would be greatly appreciated!
One thing we could do on the JDK end to reduce fragility somewhat in
this area is to provoke initialization of
sun.security.util.SecurityConstants before installing the first SM.
/Claes
On 2019-08-16 12:40, Peter Firmstone wrote:
Hello Alan,
Yes, we are aware of those issues.
I mean documenting that system Permission classes should be loaded
before setting a custom SecurityManager, accessing the file system is
important, so if you haven't loaded the necessary classes before
setting a custom SecurityManager, it won't be gracefull... The
simplest way of ensuring classes are loaded is by creating object
instances of them.
Perhaps just a note to beware of ensuring necessary classes are
loaded and let developers figure out what they need.
The recursive calls are easy enough to deal with to avoid any stack
overflows using ThreadLocal.
inTrustedCodeRecursiveCall.set(Boolean.TRUE);
try {
delegateContext = AccessController.doPrivileged(
new PrivilegedAction(){
public AccessControlContext run() {
return new
AccessControlContext(finalExecutionContext, dc);
}
}
);
}finally {
inTrustedCodeRecursiveCall.set(Boolean.FALSE); //
Must always happen, no matter what.
}
We've only really been caught out once with some jvm bootstrap
changes, otherwise it's been rock solid.
The other thing we do is once we've got more than three
ProtectionDomains on the stack is execute the ProtectionDomain
implies checks in parallel. Really helps when you're making a lot of
network calls.
Cheers,
Peter.
On 16/08/2019 5:04 PM, Alan Bateman wrote:
On 15/08/2019 23:20, Peter Firmstone wrote:
:
The following code is included in the constructor of our
SecurityManager implementation, I suspect we may need to add some
classes to this list, perhaps this is something that needs
documenting?
The checkPermission method of custom security manager can run
arbitrary code so recursive initialization, stack overflow,
bootstrap method errors, ... are always hazards. I don't know what
documentation you have in mind but I don't think there is a definite
list of classes that need to be loaded/initialized before the custom
security manager is set because it will come down to the code that
it executes in its checkPermission method.
-Alan.
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Hello Alan,
Yes, we are aware of those issues.
I mean documenting that system Permission classes should be loaded
before setting a custom SecurityManager, accessing the file system is
important, so if you haven't loaded the necessary classes before setting
a custom SecurityManager, it won't be gracefull... The simplest way of
ensuring classes are loaded is by creating object instances of them.
Perhaps just a note to beware of ensuring necessary classes are loaded
and let developers figure out what they need.
The recursive calls are easy enough to deal with to avoid any stack
overflows using ThreadLocal.
inTrustedCodeRecursiveCall.set(Boolean.TRUE);
try {
delegateContext = AccessController.doPrivileged(
new PrivilegedAction(){
public AccessControlContext run() {
return new
AccessControlContext(finalExecutionContext, dc);
}
}
);
}finally {
inTrustedCodeRecursiveCall.set(Boolean.FALSE); // Must
always happen, no matter what.
}
We've only really been caught out once with some jvm bootstrap changes,
otherwise it's been rock solid.
The other thing we do is once we've got more than three
ProtectionDomains on the stack is execute the ProtectionDomain implies
checks in parallel. Really helps when you're making a lot of network calls.
Cheers,
Peter.
On 16/08/2019 5:04 PM, Alan Bateman wrote:
On 15/08/2019 23:20, Peter Firmstone wrote:
:
The following code is included in the constructor of our
SecurityManager implementation, I suspect we may need to add some
classes to this list, perhaps this is something that needs documenting?
The checkPermission method of custom security manager can run
arbitrary code so recursive initialization, stack overflow, bootstrap
method errors, ... are always hazards. I don't know what documentation
you have in mind but I don't think there is a definite list of classes
that need to be loaded/initialized before the custom security manager
is set because it will come down to the code that it executes in its
checkPermission method.
-Alan.
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Hello Alan, This is related to URL and CodeSource and might be worth making a note of for future reference. Our software uses delayed dynamically assigned permissions via a policy provider, but for privileged domains that have AllPermission we make sure to assign this up front (We also utilise an RFC3986URIClassLoader and override CodeSource, so we're using our RFC3986 compliant URI instead of URL). The former because we dynamically download CodeSource's and there's no way of predicting up front which will be downloaded and the latter as a performance optimisation of ProtectionDomain. So we have a RFC3986 URI implementation, similar to Java's URI, it is not Serializable for security reasons. In addition to RFC3896 normalization, we have also recently added the ability to normalize IPv6 address conformant to "RFC 5952 A Recommendation for IPv6 Address Text Representation." The class also normalizes file system paths in a platform dependant manner, eg Upper Case for MS Windows, but not Unix. We have a URI::implies method that is similar to CodeSource::implies, with matching rules. We do this to avoid DNS calls or accessing the file system unnecessarily. Also, to avoid synchronization / locking overhead of PermissionCollection's and Permissions, we have a Policy provider that generates a thread confined Permissions and PermissionCollection instances on demand, allowing them to be garbage collected as soon as the implies call returns (Permission objects are initialized up front and effectively immutable and cached) a PermissionComparator also arranges the Permissions in an order that improves performance wen creating PermissionCollection instances. Our Security overhead is less than 1% as a result and the delays and blocking we had due to DNS calls have been eliminated. Regards, Peter. On 15/08/2019 8:56 PM, Alan Bateman wrote: On 15/08/2019 11:03, Claes Redestad wrote: Hi, by resolving permissions for code source URLs lazily, we can reduce early class loading during bootstrap, which improves footprint, startup and reduces the typical bootstrap dependency graph. Bug:https://bugs.openjdk.java.net/browse/JDK-8229773 Webrev: http://cr.openjdk.java.net/~redestad/8229773/webrev.00/ : I think the approach is okay as URL::openConnection doesn't actually open the connection to the resource and the creation of the URLStreamdHandler shouldn't depend on permissions granted to the caller. If a handler needs permissions when creating the URLConnection then it should do so in a privileged block. I think it would be a bit cleaner if the supporting class would lazily add the permissions for a CodeSource to the collection. That is, create it with a permissions collection and code source rather than a URL to match SecureClassLoader::getPermissions. You could potentially use it in URLClassLoader getPermission(CodeSource) method too. In System.setSecurityManager then you might need DefaultFileSystemProvider.theFileSystem() to ensure that the default file system is fully initialized before setting the SM. A minor nit this adds a spurious import BuiltinClassLoader. Also it can import the sun.security.uti class to be consistent with the existing code. -Alan
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Hello Claes,
The following code is included in the constructor of our SecurityManager
implementation, I suspect we may need to add some classes to this list,
perhaps this is something that needs documenting?
Regards,
Peter.
/* The following ensures the classes we need are loaded early to avoid
* class loading deadlock during permission checks */
try {
checkPermission(new RuntimePermission("setIO"),
SMPrivilegedContext);
} catch (ExceptionInInitializerError er){
Error e = new ExceptionInInitializerError(
"All domains on stack when starting a security manager must
have AllPermission");
e.initCause(er);
throw e;
}
constructed = Security.class != null;
On 15/08/2019 8:03 PM, Claes Redestad wrote:
Hi,
by resolving permissions for code source URLs lazily, we can reduce
early class loading during bootstrap, which improves footprint, startup
and reduces the typical bootstrap dependency graph.
Bug:https://bugs.openjdk.java.net/browse/JDK-8229773
Webrev: http://cr.openjdk.java.net/~redestad/8229773/webrev.00/
A note on the System.java changes:
The laziness makes it so that FilePermission isn't initialized
eagerly, which has an implicit bootstrap dependency on that the default
file system provider has been initialized before a SecurityManager has
been installed (since initializing FilePermission will otherwise do
recursive calls into FilePermission). We already force load of the image
reader on SecurityManager due similar bootstrap issues, which
transitively loads the DefaultFileSystemProvider.instance(), but
explicitly adding the dependency on the file system provider to
System::setSecurityManager seems prudent: it's effectively a no-op on
jdk/jdk right now, but documents the dependency and safeguards against
future implementation changes to image reader subsystem.
A note on the HelloClasslist changes:
The patch drops a number of classes from being loaded on typical
bootstrap and small apps, including the HelloClasslist tool. Since the
HelloClasslist establishes what's to be included in the default CDS
archive this can lead to a small regression on apps which actually do
use the (nio) file system provider, so explicitly adding it in avoids
tiny regression on those while not diminishing the speed-up to other
apps.
Testing: tier1-3
Thanks!
/Claes
Re: RFR: 8229773: Resolve permissions for code source URLs lazily
Hi Roger, +1 for writeReplace Personally I'd like to see some security classes break backward compatibility and remove support for serialization as it allows someone to get references to internal objects, especially since these classes are cached by the JVM. Which makes PermissionCollection.setReadOnly() very easy to bypass, by adding permissions to internal collections once you have a reference to them. Does anyone have any use cases for serializing these objects? These objects are easy to re-create by sending or recieving and parsing strings, because they are built from text based policy files, and when you do that, you are validating input, so I never did fully understand why they were made serializable. Regards, Peter. On 16/08/2019 12:54 AM, Roger Riggs wrote: Hi Claes, I would recommend using writeReplace to serialize the PermissionCollection so the serialized form does not change. Though these are unlikely to be serialized, it will be less likely to trigger some interoperability issue between different version. It may need to be documented that serializing/deserializing does not retain the lazyness. Roger On 8/15/19 10:30 AM, Claes Redestad wrote: Hi Sean, On 2019-08-15 15:07, Sean Mullan wrote: Hi Claes, I already reviewed an earlier version of this and this is pretty similar. I did have a question about whether the default serialization was ok - did you look into that more? ah, yes.. all the constituents are serializable (whether we wrap the CodeSource or its URL) so the new Lazy..Collection should be too. I was entertaining the idea that we could writeReplace the lazy collection with a PermissionCollection to avoid polluting serialization use cases with new type, but couldn't think of any real upside to that. /Clae
Re: RFR (12): 8191053: Provide a mechanism to make system's security manager immutable
reates recursive calls in the security manager implementation. The simplest fix would be to move the instantiation of SecurityManager to a static initializer in Launcher that occurs after the construction of Launcher and setting of the static launcher field. This didn't become a problem until recently upgrading to the latest Java 8 JDK, so not sure of the history here. Regards Peter. More comments inline below: Hi, On 27/03/2018 14:06, Alan Bateman wrote: >/ Moving this to security-dev. />/ />/ From the stack trace, it looks like you are using JDK 8 or older. There />/ are several changes in JDK 9 and newer in the PolicyFile code to how it />/ loads its resources that may help with the issues you are seeing. />/ />/ -Alan / [snip] >/ [java] at java.util.logging.Logger.log(Logger.java:788) />/ [java] at org.apache.river.api.security.ConcurrentPolicyFile$2.run(ConcurrentPolicyFile.java:496) / In what logging is concerned, it's probably not a good idea to use java.util.logging in a Policy/SecurityManager implementation supplied on the command line as java.util.logging uses doPrivileged and checks for permissions. Work arounds for that problem: * Delegate logging to an executor. * Use a thread local variable to detect trusted recursive calls. * Only log messages during startup, when the policy or security manager isn't active. For the record the line that throws in the first stack trace seems to be this one at LogManager.java:965 Class clz = ClassLoader.getSystemClassLoader().loadClass(word); The exception is caught and printed on System.err at line 981 allowing the caller to proceed - so it's not what is causing the ExceptionInInitializerError, but it shows that ClassLoader.getSystemClassLoader() is probably returning null at this point, and it looks like it is the same issue you're seeing at ResourceBundle.java:502 later on, which prevents the CombinerSecurityManager to initialize. Hopes this helps, -- daniel >/ />/ On 27/03/2018 13:56, Peter Firmstone wrote: />>/ Not sure if this is the right place to mention this. />>/ />>/ Anyone notice that specifying a custom security manager at jvm start />>/ up causes issues with service providers loading? If using the sun />>/ PolicyFile implementation, the policy doesn't load due to the provider />>/ failure, I have a custom policy implementation that will allow the jvm />>/ to run in this state, and other providers are also not loading, such />>/ as the logger and JCE. />>/ />>/ Note that it doesn't occur if the security manager is set />>/ programmatically in the main method at start up, only if it's set via />>/ command line option. />>/ />>/ Examples of providers not loading: />>/ />>/ [java] java.lang.NullPointerException />>/ [java] Can't load log handler "java.util.logging.ConsoleHandler" />>/ [java] java.lang.NullPointerException />>/ [java] java.lang.NullPointerException />>/ [java] at />>/ java.util.logging.LogManager$5.run(LogManager.java:965) />>/ [java] at java.security.AccessController.doPrivileged(Native />>/ Method) />>/ [java] at />>/ java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:958) />>/ [java] at />>/ java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:1578) />>/ />>/ [java] at />>/ java.util.logging.LogManager.access$1500(LogManager.java:145) />>/ [java] at />>/ java.util.logging.LogManager$RootLogger.accessCheckedHandlers(LogManager.java:1667) />>/ />>/ [java] at java.util.logging.Logger.getHandlers(Logger.java:1777) />>/ [java] at java.util.logging.Logger.log(Logger.java:735) />>/ [java] at java.util.logging.Logger.doLog(Logger.java:765) />>/ [java] at java.util.logging.Logger.log(Logger.java:788) />>/ [java] at />>/ org.apache.river.api.security.ConcurrentPolicyFile$2.run(ConcurrentPolicyFile.java:496) />>/ />>/ [java] at />>/ org.apache.river.api.security.ConcurrentPolicyFile$2.run(ConcurrentPolicyFile.java:469) />>/ />>/ [java] at java.security.AccessController.doPrivileged(Native />>/ Method) />>/ [java] at />>/ org.apache.river.api.security.ConcurrentPolicyFile.readPoliciesNoCheckGuard(ConcurrentPolicyFile.java:468) />>/ />>/ [java] at />>/ org.apache.river.api.security.ConcurrentPolicyFile.readPolicyPermissionGrants(ConcurrentPolicyFile.java:243) />>/ />>/ [java]
Re: -Djava.security.manager=problems for service providers
I tested the JDK 9 pre releases and didn't experience issues, will have to test again against the latest. Note on JDK1.8.0_162 it doesn't only affect the PolicyFile provider. Thanks, Peter. On 27/03/2018 11:06 PM, Alan Bateman wrote: Moving this to security-dev. From the stack trace, it looks like you are using JDK 8 or older. There are several changes in JDK 9 and newer in the PolicyFile code to how it loads its resources that may help with the issues you are seeing. -Alan On 27/03/2018 13:56, Peter Firmstone wrote: Not sure if this is the right place to mention this. Anyone notice that specifying a custom security manager at jvm start up causes issues with service providers loading? If using the sun PolicyFile implementation, the policy doesn't load due to the provider failure, I have a custom policy implementation that will allow the jvm to run in this state, and other providers are also not loading, such as the logger and JCE. Note that it doesn't occur if the security manager is set programmatically in the main method at start up, only if it's set via command line option. Examples of providers not loading: [java] java.lang.NullPointerException [java] Can't load log handler "java.util.logging.ConsoleHandler" [java] java.lang.NullPointerException [java] java.lang.NullPointerException [java] at java.util.logging.LogManager$5.run(LogManager.java:965) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:958) [java] at java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:1578) [java] at java.util.logging.LogManager.access$1500(LogManager.java:145) [java] at java.util.logging.LogManager$RootLogger.accessCheckedHandlers(LogManager.java:1667) [java] at java.util.logging.Logger.getHandlers(Logger.java:1777) [java] at java.util.logging.Logger.log(Logger.java:735) [java] at java.util.logging.Logger.doLog(Logger.java:765) [java] at java.util.logging.Logger.log(Logger.java:788) [java] at org.apache.river.api.security.ConcurrentPolicyFile$2.run(ConcurrentPolicyFile.java:496) [java] at org.apache.river.api.security.ConcurrentPolicyFile$2.run(ConcurrentPolicyFile.java:469) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at org.apache.river.api.security.ConcurrentPolicyFile.readPoliciesNoCheckGuard(ConcurrentPolicyFile.java:468) [java] at org.apache.river.api.security.ConcurrentPolicyFile.readPolicyPermissionGrants(ConcurrentPolicyFile.java:243) [java] at org.apache.river.api.security.ConcurrentPolicyFile.(ConcurrentPolicyFile.java:253) [java] at org.apache.river.api.security.ConcurrentPolicyFile.(ConcurrentPolicyFile.java:226) [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:154) [java] at org.apache.river.api.security.CombinerSecurityManager.(CombinerSecurityManager.java:133) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:137) [java] at org.apache.river.tool.SecurityPolicyWriter.(SecurityPolicyWriter.java:162) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [java] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [java] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [java] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [java] at java.lang.Class.newInstance(Class.java:442) [java] at sun.misc.Launcher.(Launcher.java:93) [java] at sun.misc.Launcher.(Launcher.java:54) [java] at java.lang.ClassLoader.initSystemClassLoader(ClassLoader.java:1451) [java] at java.lang.ClassLoader.getSystemClassLoader(ClassLoader.java:1436) [java] Error occurred during initialization of VM [java] java.lang.ExceptionInInitializerError [java] at java.util.ResourceBundle.getLoader(ResourceBundle.java:482) [java] at java.util.ResourceBundle.getBundle(ResourceBundle.java:783) [java] at sun.security.util.ResourcesMgr$1.run(ResourcesMgr.java:47) [java] at sun.security.util.ResourcesMgr$1.run(ResourcesMgr.java:44) [java] at java.security.AccessController.doPrivileged(Native Method) [java] at sun.security.util.ResourcesMgr.getString(ResourcesMgr.java:43) [java] at sun.security.provider.PolicyFile.addGrantEntry(PolicyFile.java:888) [java] at sun.security.provider.PolicyFile.init(PolicyFile.java:626) [java] at sun.security.provider.PolicyFile.access$400(PolicyFile.java:258) [java] at sun.security.provi
Re: Strange test failure when referencing a class in a deprivileged module
I discovered these bugs after writing a non blocking caching security manager and a couple of high scaling policy providers that combine immutability, atomic policy refresh and local variable confined mutibility. The policy provider avoids unnecessary DNS calls by replacing URL with an rfc3986 compliant uri, which uses bitshift case conversion during normalisation. I could donate parts of it that I've written under GPL, but would need to determine and contact all other authors to donate the entire works. Needless to say Java's AccessController or AccessControlContext, are often incorrectly criticised for performance issues that originate in the policy provider. The rfc3986 compliant Uri class has an api identical to java's URI, although it isn't Serializable, it could be used to test the impact of rfc3986 on java platform libraries. Cheers, Peter. Sent from my Samsung device. Include original message Original message From: Peter Firmstone Sent: 08/07/2016 05:53:47 pm To: WeijunWang Cc: jigsaw-dev ; OpenJDK Subject: Re: Strange test failure when referencing a class in a deprivileged module Yes, I've come across this before, it will occur if you write a custom security manager or policy provider and either are in force before all their required classes have been loaded. SM implementors also need to be careful of Permission checks that require another permission check, as these create recursive calls that can become infinite, but this won't occur in your simple test case. Regards, Peter. Sent from my Samsung device. Include original message Original message From: Weijun Wang Sent: 08/07/2016 01:18:56 pm To: Peter Firmstone Cc: SeanMullan ; jigsaw-dev ; OpenJDK Subject: Re: Strange test failure when referencing a class in a deprivileged module Mystery solved or problem solved? Have you fixed it somewhere else? Thanks Max On 7/7/2016 17:00, Peter Firmstone wrote: > Problem solved, even though it didn't occur on Java 8, the potential for > it to occur still exists there, it's simply that Java 9 seems to have > hit this execution path, it was a latent bug. > > Cheers, > > Peter.
Re: Strange test failure when referencing a class in a deprivileged module
Yes, I've come across this before, it will occur if you write a custom security manager or policy provider and either are in force before all their required classes have been loaded. SM implementors also need to be careful of Permission checks that require another permission check, as these create recursive calls that can become infinite, but this won't occur in your simple test case. Regards, Peter. Sent from my Samsung device. Include original message Original message From: Weijun Wang Sent: 08/07/2016 01:18:56 pm To: Peter Firmstone Cc: SeanMullan ; jigsaw-dev ; OpenJDK Subject: Re: Strange test failure when referencing a class in a deprivileged module Mystery solved or problem solved? Have you fixed it somewhere else? Thanks Max On 7/7/2016 17:00, Peter Firmstone wrote: > Problem solved, even though it didn't occur on Java 8, the potential for > it to occur still exists there, it's simply that Java 9 seems to have > hit this execution path, it was a latent bug. > > Cheers, > > Peter.
Re: Strange test failure when referencing a class in a deprivileged module
Can you change the test to call checkPermission prior to setting SSL as the system security manager? This will just ensure that all necessary classes are loaded prior to SSL becoming the security manager. Regards, Peter. Sent from my Samsung device. Include original message Original message From: Wang Weijun Sent: 07/07/2016 01:45:08 pm To: Peter Firmstone Cc: SeanMullan ; jigsaw-dev ; OpenJDK Subject: Re: Strange test failure when referencing a class in a deprivileged module I run the program directly (not from jtreg) and the result is different: Exception in thread "main" java.lang.BootstrapMethodError: call site initialization exception at SSL.checkPermission(SSL.java:21) at java.lang.SecurityManager.checkPropertyAccess(java.base@9-ea/SecurityManager.java:1285) at java.lang.System.getProperty(java.base@9-ea/System.java:762) at SSL.main(SSL.java:28) Then I add -Djava.security.debug=all, and the exact old exception is back! The debug output [1] has nothing special, in fact there is no difference if I substitute SQLPermission to AllPermission, except that with AllPermission the program runs fine. Thanks Max [1] https://gist.github.com/wangweij/b7d031dea47c456b169f1e733eac > On Jul 7, 2016, at 11:02 AM, Peter Firmstone >wrote: > > Have you tried setting the jvm in security debug mode to see else might be >occurring? If so can you send the output? > > Regards, > > Peter. > > Sent from my Samsung device.
Re: Strange test failure when referencing a class in a deprivileged module
Problem solved, even though it didn't occur on Java 8, the potential for it to
occur still exists there, it's simply that Java 9 seems to have hit this
execution path, it was a latent bug.
Cheers,
Peter.
Sent from my Samsung device.
Include original message
Original message
From: Wang Weijun
Sent: 07/07/2016 06:55:26 pm
To: Peter Firmstone
Cc: SeanMullan ; jigsaw-dev
; OpenJDK
Subject: Re: Strange test failure when referencing a class in a deprivileged
module
> On Jul 7, 2016, at 4:47 PM, Peter Firmstone
>wrote:
>
> Sorry, I wasn't clear, I meant, create a new instance of SSL, call its
>checkPermission method, then make that instance your system security manager.
That would cure the test too, since it references SQLPermission when
checkPermission is called.
In another reply to Alan, I said a single "new SQLPermission("setLog");" is
enough.
--Max
>
> Although the check permission call will be repeated in the test, it will have
>run while the standard SecurityManager is in force, so the jvm won't lazy load
>any classes that require permission checks, during the test permission check
>as that will happen during the first permission check call.
>
> Cheers,
>
> Peter.
>
> Sent from my Samsung device.
>
> Original message
> From: Wang Weijun
> Sent: 07/07/2016 06:27:43 pm
> To: Peter Firmstone
> Cc: SeanMullan ; jigsaw-dev
>; OpenJDK
> Subject: Re: Strange test failure when referencing a class in a deprivileged
>module
>
> Like this?
>
> https://gist.github.com/wangweij/6992aaf9617b9e2f242ddf9e391ea5f6
>
> The horizontal line is printed on line 706 of the gist.
>
> --Max
>
> > On Jul 7, 2016, at 12:57 PM, Peter Firmstone
>wrote:
> >
> > Can you change the test to call checkPermission prior to setting SSL as the
>system security manager?
> >
> > This will just ensure that all necessary classes are loaded prior to SSL
>becoming the security manager.
> >
> > Regards,
> >
> > Peter.
>
>
Re: Strange test failure when referencing a class in a deprivileged module
Yes, that's correct. ;) Sent from my Samsung device. Include original message Original message From: Alan Bateman Sent: 07/07/2016 06:37:49 pm To: Wang Weijun Cc: jigsaw-dev ; OpenJDK Subject: Re: Strange test failure when referencing a class in a deprivileged module On 07/07/2016 09:27, Wang Weijun wrote: > Like this? > > https://gist.github.com/wangweij/6992aaf9617b9e2f242ddf9e391ea5f6 > > The horizontal line is printed on line 706 of the gist. > I assume the suggestion was to reference SQLPermission before setting the security manager. As regards module loading then we can re-wording this specific case but fundamentally then anything in the checkPermission implementation that triggers a permission check is going to lead to a recursive call to checkPermission. -Alan
Re: Strange test failure when referencing a class in a deprivileged module
Sorry, I wasn't clear, I meant, create a new instance of SSL, call its checkPermission method, then make that instance your system security manager. Although the check permission call will be repeated in the test, it will have run while the standard SecurityManager is in force, so the jvm won't lazy load any classes that require permission checks, during the test permission check as that will happen during the first permission check call. Cheers, Peter. Sent from my Samsung device. Include original message Original message From: Wang Weijun Sent: 07/07/2016 06:27:43 pm To: Peter Firmstone Cc: SeanMullan ; jigsaw-dev ; OpenJDK Subject: Re: Strange test failure when referencing a class in a deprivileged module Like this? https://gist.github.com/wangweij/6992aaf9617b9e2f242ddf9e391ea5f6 The horizontal line is printed on line 706 of the gist. --Max > On Jul 7, 2016, at 12:57 PM, Peter Firmstone >wrote: > > Can you change the test to call checkPermission prior to setting SSL as the >system security manager? > > This will just ensure that all necessary classes are loaded prior to SSL >becoming the security manager. > > Regards, > > Peter.
Re: Strange test failure when referencing a class in a deprivileged module
Have you tried setting the jvm in security debug mode to see else might be
occurring? If so can you send the output?
Regards,
Peter.
Sent from my Samsung device.
Include original message
Original message
From: Wang Weijun
Sent: 07/07/2016 11:08:58 am
To: Peter Firmstone
Cc: SeanMullan ; jigsaw-dev
; OpenJDK
Subject: Re: Strange test failure when referencing a class in a deprivileged
module
> On Jul 7, 2016, at 8:19 AM, Peter Firmstone
>wrote:
>
> Perhaps the policy provider hasn't been refreshed when the new security
>manager is in force? Try doing a Policy.refresh() in the SecurityManager
>constructor.
You mean Policy.getPolicy().refresh()? No difference
And I don't think this simple SecurityManager makes use of Policy at all.
Thanks
Max
>
> Regards,
>
> Peter.
>
> Sent from my Samsung device.
>
> Original message
> From: Wang Weijun
> Sent: 07/07/2016 09:47:09 am
> To: Sean Mullan
> Cc: jigsaw-dev ; OpenJDK
>
> Subject: Re: Strange test failure when referencing a class in a deprivileged
>module
>
>
> > On Jul 7, 2016, at 5:04 AM, Sean Mullan wrote:
> >
> > Does your SSL code match up with the stack trace? The test only has 27
>lines, but the stack trace says it was called from line 42.
>
> My local SSL.java still contains the GPL comments and I didn't paste them
>here. Line 42 is
>
> if (!(perm instanceof SQLPermission)) {
>
> BTW, I also tried putting the SecurityManager impl into a different codePath
>with AllPermission granted, and load it on the command line with
>-Djava.security.manager=X, the test still failed with the same exception.
>
> --Max
>
>
Re: Strange test failure when referencing a class in a deprivileged module
Perhaps the policy provider hasn't been refreshed when the new security manager
is in force? Try doing a Policy.refresh() in the SecurityManager constructor.
Regards,
Peter.
Sent from my Samsung device.
Include original message
Original message
From: Wang Weijun
Sent: 07/07/2016 09:47:09 am
To: Sean Mullan
Cc: jigsaw-dev ; OpenJDK
Subject: Re: Strange test failure when referencing a class in a deprivileged
module
> On Jul 7, 2016, at 5:04 AM, Sean Mullan wrote:
>
> Does your SSL code match up with the stack trace? The test only has 27 lines,
>but the stack trace says it was called from line 42.
My local SSL.java still contains the GPL comments and I didn't paste them here.
Line 42 is
if (!(perm instanceof SQLPermission)) {
BTW, I also tried putting the SecurityManager impl into a different codePath
with AllPermission granted, and load it on the command line with
-Djava.security.manager=X, the test still failed with the same exception.
--Max
Re: The future of Serialization
Interesting, language features for modules, won't necessarily involve ClassLoader's (my assumptions were based on existing systems) although you'd expect modules to have their own ProtectionDomain. An alternative to isolates, is separate processes with jvm class sharing enabled. I'll keep an eye out for the JSR. When is a better timeframe, roughly, to discuss Serializable? Regards, Peter. - Original message - > On 11/08/2014 13:06, Peter Firmstone wrote: > > Thanks Alan, I can relate to time poverty :) > > > > I might be assuming too much, but if there's interest in doing > > something with Serialization, I'd be interested in learning about > > plans or difficulties involved in deserialization and modules. It can > > be a little more difficult to find the correct ClassLoader to resolve > > classes during deserialization when ClassLoader relationships aren't > > hierarchial. Object streams can be annotated with module information > > to assist resolution. > The issues around visibility when deserializing are somewhat independent > of modules. The usual way to deal with such matters is to override the > resolveClass method. Another long standing suggestion is for > ObjectInputStream to define a new constructor that takes a class loader > to avoid the stack walk to get the user-defined loader. It remains to > seen but if we can avoid changing visibility then we don't change the > status quo. > > : > > > > Got any links to info on extending access control rules? > Not yet, a future JSR will define the standard module system and there > will be a corresponding JEP for the implementation. > > -Alan.
Re: The future of Serialization
Thanks Alan, I can relate to time poverty :) I might be assuming too much, but if there's interest in doing something with Serialization, I'd be interested in learning about plans or difficulties involved in deserialization and modules. It can be a little more difficult to find the correct ClassLoader to resolve classes during deserialization when ClassLoader relationships aren't hierarchial. Object streams can be annotated with module information to assist resolution. On the subject of isolates, I found Ribbons interesting: https://www.cs.purdue.edu/homes/peugster/Ribbons/RJ.pdf https://www.cs.purdue.edu/homes/peugster/Ribbons/ Got any links to info on extending access control rules? Regards, Peter. On 11/08/2014 9:21 PM, Alan Bateman wrote: On 09/08/2014 06:56, Peter Firmstone wrote: I've noticed there's not much interest in improving Serialization on these lists. This makes me wonder if java Serialization has lost relevance in recent years with the rise of protocol buffers apache thrift and other means of data transfer over byte streams. Just to add to Brian's comments, I think part of it is that many people are busy with other things, preparing for JDK 9 for example. So I think there is a lot of support for investigation and proposals that would improve things, it's just that some people are too busy to respond. I don't know if isolates will be included with JDK 9 for Jigsaw, or whether ClassLoaders alone will provide isolation for modules. The ability to limit visibility and provide isolation of implementation classes as well as providing limits on memory and threads for isolated modules would also improve platform security. If by "isolates" you mean JSR 121 then I think that would be well beyond the scope, as would resource management. This isn't really the thread to discuss how module boundaries will work but just to say that class loaders and visibility can be weak when it comes to module boundaries. There are other options available, particularly when the ability to extend the access control rules are on the table. So I would suggest not making any assumptions here for now. -Alan.
Re: The future of Serialization
On 11/08/2014 8:12 PM, Peter Firmstone wrote:
Brian,
Thanks for picking up on my frustration ;)
I have something in mind for Serializable2 to address cyclic data
structures and the possibility of independant evolution of super and
child classes, while retaining a relatively clean public api, with one
optional private method. The methods and interfaces proposed are
suitable for any alternative ObjectInput and ObjectOutput implementation.
An interface exists in Apache River, it's called Startable, it has one
method:
public void start() throws Exception;
It's called by a framework to allow an Object to start threads,
publish "this" or throw an exception after construction. The intent
is to allow an object to be immutable with final fields and be
provided with a thread of execution after construction and before
publication.
Something similar can be used to wire up circular relations, let met
explain:
Every class that implements Serializable has one thing in common, the
Serialization protocol and every Object instance of a Serializable
class has an arbitrary serial form.
I propose a final class representing SerialForm for an object, that
cannot be extended, requires privilege to instantiate and also
performs method guard security checks, for all callers with the
exception of a calling class reading or writing its own serial form.
SerialForm needs a parameter field key identity represented by the
calling Class,
Sorry, that should read "field name", not "method name".
the method name and the field's Class type, this key would be used for
both writing and retrieving a field entry in SerialForm. SerialForm
will also provide a method to advise if a field key contains a
circular relation, any field entry in SerialForm that would contain a
circular relation is not populated until after construction of the
current object is complete.
An arbitrary Serializable2 Object instance may be composed of a
hierarchy of classes, each belonging to a separate ProtectionDomain.
For the following interface:
public interface Serializable2 {
void writeObject(SerialForm serial) throws IOException;
}
Implementers of Serializable2 must:
1. Implement writeObject
2. Implement a constructor with the signature: (SerialForm serial).
Implementors that need to check invariants, delay throwing an
Exception, publish "this" or set a circular reference after
construction should:
4. Implement: private void readObjectNoData() throws
InvalidObjectException;
Child class implementations should:
5. Call their super class writeObject method and superclass
constructor, but may call any super class constructor or methods.
Compatibility and Evolution:
1. Fields can be included or omitted from SerialForm, by an
implementation, without breaking compatibility, provided a null
reference is accepted during deserialization.
2. Child classes in a hierarchy; all Serializable2 implementing
superclass constructors have the same signature; the superclass
implementation can be substituted, without breaking child class
deserialization (provided this is the constructor used by the
child class).
3. There is no serialVersionUID.
4. Child class Serializable2 implementations can extend a superclass
without a zero arg constructor that doesn't itself implement
Serializable2.
5. Child classes that do not override writeObject will not be
serialized, so can effectively opt out.
6. Because implementations are required to implement public methods,
there is no "Magic".
7. Serializable2 shouldn't extend Serializable, allowing classes to
implement both interfaces for a period of time (for that reason
the signature for readObjectNoData may need to be changed for
Serializable2).
8. ObjectInputStream and ObjectOutputStream can be extended to
support both implementations for compatibility, however
alternative stream implementations would be preferable for
Serializable2 to avoid Serializable security issues. The new
implementations should be possible to substitute because both
types would use the same Stream Protocol, provided the classes
being deserialized implement Serializable2.
My reasoning for retaining readObjectNoData() and for updating field
entry's in SerialForm that contain circular relations after
construction, is:
1. An object reference for the object currently being deserialized
can be passed to another object's constructor (via a SerialForm
instance) after the current Object's constructor completes,
allowing safe publication of final field freezes that occur at the
end of construction.
2. When the Serialization2 Framework becomes aware of an object that
contains a circular relationship while that object is in the
process of being deserialized, the seco
Re: The future of Serialization
Brian,
Thanks for picking up on my frustration ;)
I have something in mind for Serializable2 to address cyclic data
structures and the possibility of independant evolution of super and
child classes, while retaining a relatively clean public api, with one
optional private method. The methods and interfaces proposed are
suitable for any alternative ObjectInput and ObjectOutput implementation.
An interface exists in Apache River, it's called Startable, it has one
method:
public void start() throws Exception;
It's called by a framework to allow an Object to start threads, publish
"this" or throw an exception after construction. The intent is to allow
an object to be immutable with final fields and be provided with a
thread of execution after construction and before publication.
Something similar can be used to wire up circular relations, let met
explain:
Every class that implements Serializable has one thing in common, the
Serialization protocol and every Object instance of a Serializable class
has an arbitrary serial form.
I propose a final class representing SerialForm for an object, that
cannot be extended, requires privilege to instantiate and also performs
method guard security checks, for all callers with the exception of a
calling class reading or writing its own serial form. SerialForm needs
a parameter field key identity represented by the calling Class, the
method name and the field's Class type, this key would be used for both
writing and retrieving a field entry in SerialForm. SerialForm will also
provide a method to advise if a field key contains a circular relation,
any field entry in SerialForm that would contain a circular relation is
not populated until after construction of the current object is complete.
An arbitrary Serializable2 Object instance may be composed of a
hierarchy of classes, each belonging to a separate ProtectionDomain.
For the following interface:
public interface Serializable2 {
void writeObject(SerialForm serial) throws IOException;
}
Implementers of Serializable2 must:
1. Implement writeObject
2. Implement a constructor with the signature: (SerialForm serial).
Implementors that need to check invariants, delay throwing an Exception,
publish "this" or set a circular reference after construction should:
4. Implement: private void readObjectNoData() throws
InvalidObjectException;
Child class implementations should:
5. Call their super class writeObject method and superclass
constructor, but may call any super class constructor or methods.
Compatibility and Evolution:
1. Fields can be included or omitted from SerialForm, by an
implementation, without breaking compatibility, provided a null
reference is accepted during deserialization.
2. Child classes in a hierarchy; all Serializable2 implementing
superclass constructors have the same signature; the superclass
implementation can be substituted, without breaking child class
deserialization (provided this is the constructor used by the
child class).
3. There is no serialVersionUID.
4. Child class Serializable2 implementations can extend a superclass
without a zero arg constructor that doesn't itself implement
Serializable2.
5. Child classes that do not override writeObject will not be
serialized, so can effectively opt out.
6. Because implementations are required to implement public methods,
there is no "Magic".
7. Serializable2 shouldn't extend Serializable, allowing classes to
implement both interfaces for a period of time (for that reason
the signature for readObjectNoData may need to be changed for
Serializable2).
8. ObjectInputStream and ObjectOutputStream can be extended to
support both implementations for compatibility, however
alternative stream implementations would be preferable for
Serializable2 to avoid Serializable security issues. The new
implementations should be possible to substitute because both
types would use the same Stream Protocol, provided the classes
being deserialized implement Serializable2.
My reasoning for retaining readObjectNoData() and for updating field
entry's in SerialForm that contain circular relations after
construction, is:
1. An object reference for the object currently being deserialized
can be passed to another object's constructor (via a SerialForm
instance) after the current Object's constructor completes,
allowing safe publication of final field freezes that occur at the
end of construction.
2. When the Serialization2 Framework becomes aware of an object that
contains a circular relationship while that object is in the
process of being deserialized, the second object will not be
instantiated until after the constructor of the first object in
the relationship completes. Data read in from the stream can be
stored in a SerialForm without requiri
The future of Serialization
I've noticed there's not much interest in improving Serialization on these lists. This makes me wonder if java Serialization has lost relevance in recent years with the rise of protocol buffers apache thrift and other means of data transfer over byte streams. The burden of implementing Serializable can significantly hamper developers efforts when refactoring, it's quite common for some projects to make no guarantee regarding Serialization compatibility between releases. Also implementation of Serializable can double project development hours, hamper future development and increase software maintenance costs. Serialization also presents opportunities for attackers and has been responsible for a number of zero day exploits. I don't know if isolates will be included with JDK 9 for Jigsaw, or whether ClassLoaders alone will provide isolation for modules. The ability to limit visibility and provide isolation of implementation classes as well as providing limits on memory and threads for isolated modules would also improve platform security. Serialization may provide a means to hot upgrade modules, but more flexible options that doesn't cause serial data lock in need to be developed. Should Serializable eventually be deprecated? Should Serialization be disabled by default? Should a new mechanism be developed? If a new mechanism is developed, what about circular object relationships? Regards, Peter.
Re: JEP Review Request: Improve Security Manager Performance
No David is right, the code will run with the privileges of it's own ProtectionDomain, so if that PD is not trusted, the code cannot bypass security checks from static initializers. This does not break secure by default. A static initializer can write to static fields, but public or protected static mutable fields are a secuity risk, with or without a privileged call. Because classes are loaded lazily by the jvm, the call stack context is randomised causing confusing and difficult to debug policy file configuration for administrators, often resulting in a relaxation of permissions beyond minimum privilege principles. Making the suggested change actually improves security. Regards, Peter. - Original message - > On 07/23/2014 07:07 AM, Tom Hawtin wrote: > > On 23/07/2014 05:26, David M. Lloyd wrote: > > > I would suggest that one or more of the following be done to mitigate > > > this problem: > > > > > > • Always have static initialization blocks be privileged (this does > > > require users to be cognizant of this fact when writing static > > > blocks) > > > > If we were following "secure by default", this would break it. It turns > > out having a static initaliser run with an unprivileged acc highlights > > code that is doing something naughty. > > I thought this mindset might dominate, which is unfortunate. In > practice, it is far better for code to be predictable, concise, and > clear. It does not really make any sense to have random security > contexts in place and then call it "secure"; it makes more sense to just > tell people "hey your static initializers are privileged". It's not > like normal directly invokable methods where the user can pass in > arguments and get return values from code that runs in a privileged > context. It is very, very unusual for a static initializer to perform > any function which is practically exploitable; they are generally > stateless already, taking no input and producing no output. I don't > think that it can be shown that the blanket ideology really provides any > measurable, real-world benefit; on the contrary, it causes real, > measurable detriment. I don't think anyone ever said "Oh, it's a good > thing I got that AccessControlException in my static initializer; now I > know that doing XYZ needed a privileged block". > > Nevertheless, I never actually expected to convince anyone of this - > it's one of those ideas which seems culturally "against the grain", i.e. > the general principle tends to outweigh the practical reality. Still, I > had to propose it, in order to be right with myself. :-) > > > > • Allow static initialization blocks to partake in the aforementioned > > > annotation-driven privileged method idea > > > > Together with the last point, this does make the elevated privilege > > contain a wider block of code than is necessary. This is [particularly > > the case with static initialisers and initialisers where the code can > > be spread throughout the class. OTOH, relevant sections of code could > > be split out into small methods. > > Exactly - in this case I would even expect that you could separately > annotate each static init with its privileges. The important thing > would still be that the compiler can do this without constructing a new > class for each chunk (or even a lambda if it could be avoided). > > > > • Introduce a new permission checking mechanism which examines only a > > > specific relevant caller's protection domain (perhaps determined by > > > filter expression, possibly using the stack examination scheme that > > > Mandy Chung has been working on) > > > > Immediate caller checking, though has similarities to link-time access > > checking, has a spectacularly unfortunate history. > > That is an observation. :-) > > > > • Introduce a programmatic "elevation" mechanism that increases the > > > privileges of the currently executing method for the remainder of its > > > execution without requiring a call-in to doPrivileged or > > > instantiation of a privileged action object > > > > Tom > > -- > - DML
Re: Policy provider comparison
There doesn't seem to be much interest in adopting an external policy provider, so I'll explain how to achieve a significant performance improvement, in case someone's interested in improving performance of the default jvm policy provider. 1. Thread confine and immediately discard PermissionCollection objects, do not cache them as is done presently. 2. We created an object representation of grant statements from policy files in memory, it is immutable and replaced during policy refresh calls, it is also non blocking and highly scalable. 3. During a policy implies call, collect all grant statements that imply a ProtectionDomain from an implies call. Take Permission objects from the grant statements and add them to PermissionCollection and Permissions (a heterogenous collection of PermissionCollection objects, itself an instance of PermissionCollection). Determine the result of the implies call and discard the Permissions, do not cache or share them with other threads. 4. Sort Permission objects before adding them to ProtectionDomain, to minimise unnecessary DNS calls from SocketPermission etc. 5. Do not add Permissions to a ProtectionDomain that delegates to the Policy provider, or Permissions may be checked twice. 6. Only add Permissions to ProtectionDomain's when they have AllPermissions or when it does not delegate to a Policy provider. PermissionCollection was designed to be threadsafe, I think this was a mistake (hindsight is 20:20), similar to HashTable and Vector, PermissionCollection instances should be thread confined (uncontended synchronization is fast). Permissions has a race condition that prevents UnresolvedPermission's from resolving on occassion too, thread confinement fixes that too. Regards, Peter. On 19/07/2014 6:31 PM, Peter Firmstone wrote: Thought I might provide a comparison using our random stress test on a 4 way sony vaio, JDK7 Windows 7 amd64: Note that using sun.security.provider.PolicyFile doubles the duration of the stress test and it consumes 73% of CPU. In comparison org.apache.river.api.security.ConcurrentPolicyFile uses 0% CPU and the test completes in half the time, with our policy provider we run at raw socket speed. See attached. Regards, Peter.
JEP Improve Security Manager Performance
Sean, Would a SecurityManager and Policy provider combination that has no perceptable impact on performance and is highly scalable qualify? It's a drop in fully compatible replacement. Regards, Peter. Original Message Subject:Re: Concurrent Policy provider Date: Thu, 10 Jul 2014 22:20:33 -0500 From: David M. Lloyd To: security-dev@openjdk.java.net Maybe you already know this, but this past April, Sean Mullan (from Oracle) posted a "JEP Review Request" [1] for a blanket initiative to improve security manager performance. This kind of thing sounds like exactly the sort of thing that would fit in under that initiative, IMO. [1] http://mail.openjdk.java.net/pipermail/security-dev/2014-April/010432.html On 07/10/2014 10:00 PM, Peter Firmstone wrote: Would there be interest in using a Policy provider and SecurityManager designed for concurrency in Java 9? Some of the issues experienced and solutions are mentioned below. http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/org/apache/river/api/security/ Regards, Peter. Original Message Subject: Concurrency-interest Digest, Vol 113, Issue 45 Date: Mon, 30 Jun 2014 12:00:01 -0400 From: concurrency-interest-requ...@cs.oswego.edu Reply-To: concurrency-inter...@cs.oswego.edu To: concurrency-inter...@cs.oswego.edu Send Concurrency-interest mailing list submissions to concurrency-inter...@cs.oswego.edu To subscribe or unsubscribe via the World Wide Web, visit http://cs.oswego.edu/mailman/listinfo/concurrency-interest or, via email, send a message with subject or body 'help' to concurrency-interest-requ...@cs.oswego.edu You can reach the person managing the list at concurrency-interest-ow...@cs.oswego.edu When replying, please edit your Subject line so it is more specific than "Re: Contents of Concurrency-interest digest..." Today's Topics: 1. Re: ThreadLocalRandom clinit troubles (Peter Firmstone) 2. Re: ThreadLocalRandom clinit troubles (Alan Bateman) -- Message: 1 Date: Mon, 30 Jun 2014 20:02:30 +1000 From: Peter Firmstone To: Peter Levart Cc: concurrency-inter...@cs.oswego.edu Subject: Re: [concurrency-interest] ThreadLocalRandom clinit troubles Message-ID:<53b135b6.1030...@zeus.net.au> Content-Type: text/plain; charset=UTF-8; format=flowed Hi Peter, There are a number of bottlenecks throughout the security infrastructure, we have reimplemented it as follows to avoid them and have also addressed some long standing issues: SecurityManager - cache previously checked AccessControlContext's (using a cache structure based on Cliff Click's non blocking hash map and Doug Lee's concurrent skip list set), to avoid repeately calling the policy provider (for example, you have numerous tasks in an ExecutorService and each has an identical AccessControlContext and all tasks cause a SocketPermission check). Policy provider - replaced the built in Java policy provider. The built in Java Policy provider will hold a lock while making DNS calls, bringing all permission checks to a grinding halt. It will also hold locks while accessing the file system. To avoid making DNS calls we've implemented our own policy provider, originated from Apache Harmony, but modified to use non blocking immutability. It creates PermissionCollection's on demand, they're not shared among threads and are ordered in the most favourable order for a fast result. It also supports undocumented Java file policy provider functionality. Because Permission objects are immutable but lazily initialized, we call getActions() to ensure their state is completely initialized prior to publication. The policy provider uses a strictly RFC 3986 compliant immutable URI class, it performs bit shift operations on ASCII strings during normalisation and is used by the policy provider to avoid making DNS calls when checking CodeBase policy file permissions. As a result, the cost of the security infrastructure is less than 1% of CPU load during stress tests. This is part of the Apache River project, JERI (Jini Extensible Remote Invocation) performs remote method invocations as tasks running in a system threadpool, each task executes concurrently on any exported service. SecureClassLoader uses CodeSource's as keys in a Map, CodeSource uses URL in equals() and hashCode(), so we also have a RFC3986 compliant URLClassLoader to avoid making unnecessary DNS calls in SecureClassLoader. Rather than rely on an external untrusted DNS to determine the identity of CodeSource's, we use RFC 3986 compliant normalisation, without making DNS calls. URL's still make DNS calls when retrieving a URL. The difference is, two different host names that previously re
Concurrent Policy provider
Would there be interest in using a Policy provider and SecurityManager designed for concurrency in Java 9? Some of the issues experienced and solutions are mentioned below. http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/org/apache/river/api/security/ Regards, Peter. Original Message Subject:Concurrency-interest Digest, Vol 113, Issue 45 Date: Mon, 30 Jun 2014 12:00:01 -0400 From: concurrency-interest-requ...@cs.oswego.edu Reply-To: concurrency-inter...@cs.oswego.edu To: concurrency-inter...@cs.oswego.edu Send Concurrency-interest mailing list submissions to concurrency-inter...@cs.oswego.edu To subscribe or unsubscribe via the World Wide Web, visit http://cs.oswego.edu/mailman/listinfo/concurrency-interest or, via email, send a message with subject or body 'help' to concurrency-interest-requ...@cs.oswego.edu You can reach the person managing the list at concurrency-interest-ow...@cs.oswego.edu When replying, please edit your Subject line so it is more specific than "Re: Contents of Concurrency-interest digest..." Today's Topics: 1. Re: ThreadLocalRandom clinit troubles (Peter Firmstone) 2. Re: ThreadLocalRandom clinit troubles (Alan Bateman) -- Message: 1 Date: Mon, 30 Jun 2014 20:02:30 +1000 From: Peter Firmstone To: Peter Levart Cc: concurrency-inter...@cs.oswego.edu Subject: Re: [concurrency-interest] ThreadLocalRandom clinit troubles Message-ID:<53b135b6.1030...@zeus.net.au> Content-Type: text/plain; charset=UTF-8; format=flowed Hi Peter, There are a number of bottlenecks throughout the security infrastructure, we have reimplemented it as follows to avoid them and have also addressed some long standing issues: SecurityManager - cache previously checked AccessControlContext's (using a cache structure based on Cliff Click's non blocking hash map and Doug Lee's concurrent skip list set), to avoid repeately calling the policy provider (for example, you have numerous tasks in an ExecutorService and each has an identical AccessControlContext and all tasks cause a SocketPermission check). Policy provider - replaced the built in Java policy provider. The built in Java Policy provider will hold a lock while making DNS calls, bringing all permission checks to a grinding halt. It will also hold locks while accessing the file system. To avoid making DNS calls we've implemented our own policy provider, originated from Apache Harmony, but modified to use non blocking immutability. It creates PermissionCollection's on demand, they're not shared among threads and are ordered in the most favourable order for a fast result. It also supports undocumented Java file policy provider functionality. Because Permission objects are immutable but lazily initialized, we call getActions() to ensure their state is completely initialized prior to publication. The policy provider uses a strictly RFC 3986 compliant immutable URI class, it performs bit shift operations on ASCII strings during normalisation and is used by the policy provider to avoid making DNS calls when checking CodeBase policy file permissions. As a result, the cost of the security infrastructure is less than 1% of CPU load during stress tests. This is part of the Apache River project, JERI (Jini Extensible Remote Invocation) performs remote method invocations as tasks running in a system threadpool, each task executes concurrently on any exported service. SecureClassLoader uses CodeSource's as keys in a Map, CodeSource uses URL in equals() and hashCode(), so we also have a RFC3986 compliant URLClassLoader to avoid making unnecessary DNS calls in SecureClassLoader. Rather than rely on an external untrusted DNS to determine the identity of CodeSource's, we use RFC 3986 compliant normalisation, without making DNS calls. URL's still make DNS calls when retrieving a URL. The difference is, two different host names that previously resolved to an identical IP address will no longer be equal, but now we can use dynamic dns addresses and fail over replication for domain names that use a range of IP addresses to answer for one domain address. Standard Java SecureClassLoader behaviour can be had by setting a system property This also reduces or avoids calls to the built in java NameServiceProvider. If this code was in the JVM libraries, we wouldn't need it in our project. This code is freely available. Regards, Peter. On 29/06/2014 7:29 PM, Peter Levart wrote: On 06/29/2014 03:53 AM, Peter Firmstone wrote: It does look like a good solution. You might wonder why we still need a custom SecurityManager? Concurrency. The ageing java security infrastructure is a huge bottleneck for multithreaded code. Regards, Peter. Hi Peter, If you could identify the most cri
