please review: Draft proposal for Class loader deadlock fix

[Karen Kinnear]

Dear Java Developers, For several years we have been hearing requests from you for a fix for the class loader problem, 4670071, in which custom class loaders can deadlock. We recognize this is an important issue for many of our customers and apologize for how long it has taken us to find a solution. We have been working on the underlying technology changes for this for several years, improving the VM class loading for JDK5, JDK6 and Hotspot 10. We now have the VM technology in place and are ready to propose minimal API changes which would allow custom class loaders to load multiple classes in parallel without deadlocking. We discussed an earlier proposal at JavaOne 2008 and believe we have improved and simplified our approach based on the very helpful feedback we received there and in subsequent communications.  Over the years, many of you have provided valuable feedback and have participated in discussions on finding a solution for this problem. Thank you. We are now pleased to announce the availability of the initial draft of the proposal for fixing the class loader deadlock problem, 4670071. We would like to invite you to review the proposal and provide us your feedback so we can make sure that the eventual solution will meet your need.  The proposal is attached to this email. We will post updates to the proposal at http://openjdk.java.net/groups/core-libs Our current goal is to make a prototype available in mid December 2008 after incorporating feedback. We have a plea for you, we could use additional test cases. We have a small number of samples that we are very grateful for and have used to test the multiple prototypes tried so far. Additional test cases could make a big difference in our understanding of any design goals or constraints that we have missed as well as in our confidence that a given solution will work. If you have any test cases, sample code which show the problems you care about, please send them to us.  We want to thank you for taking the time to help us resolve this challenging problem, and thank you for your patience. Please send your feedback and comments to this mailing list or to any of the following development engineers.  Karen Kinnear, VM runtime technical lead -  [EMAIL PROTECTED]  Valerie Peng, VM core libraries - [EMAIL PROTECTED]  Jeff Nisewanger, VM core libraries - [EMAIL PROTECTED]

Title: ClassLoaderForTest < HotspotRuntime < JavaSE

Class Loader API Modifications for Deadlock Fix Version .01: November 13, 2008 Authors: Valerie Peng, Jeff Nisewanger, Karen Kinnear Audience: Community Review Goals The goal of Class Loader API modifications is to resolve a critical customer issue: it is easy to deadlock custom class loaders that do not adhere to a strict hierarchical class loader delegation model. Non-Goals Other Class Loader RFEs that require API modification will be addressed via individual RFEs, separately from this fix for the deadlock problem. This proposal does not address any re-architecture for performance Motivation Top webbug: 817 SDN votes April 2008 4670071 java.lang.ClassLoader.loadClassInternal(String) is too restrictive Causing problems for key customers with no good workaround that works for everyone. Technical Design Constraints Must allow non-hierarchical class delegation topologies without deadlocks Must be backwards compatible Existing class loaders must continue to work unchanged Existing code that invokes a class loader directly via loadClass(String) or a class loader that calls loadClass(String, boolean) must continue to work unchanged Must continue to support the temporary risky flag combination -XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClass The intention is to deprecate the -XX:+UnsyncloadClass flag when customers adopt the new mechanism. Common workaround Right now some customers work around the deadlock by explicitly issuing a wait() on the class loader lock. While this has not been sufficient to solve many customers problems, we need to continue to support this until those customers have an opportunity to migrate to the new mechanism. Must be possible to define guidance to migrate custom class loaders to take advantage of asynchronous behavior Today, with the class loader lock held around class loader operations, class loaders are not required to handle asynchronous multi-threaded class loading. A mechanism which avoids deadlock will require multi-thread safety awareness. Given that supporting multi-threaded class loading requires careful thought and planning, the new mechanism must provide a way for a class loader to explicitly claim that they are multi-thread safe. Therefore: the fix for these bugs will require code changes by custom class loader authors if their class loaders are candidates for deadlock. Given that each class loader that chooses to adopt the new mechanism must be explicitly safe for concurrent class loading by multiple threads, the claim that a class loader is multi-thread safe must not be automatically inherited. Class loaders that adopt the new mechanism must be able to run on older JREs. So, for example, the design must allow for a way to detect if the mechanism exists, and can not count on inheritance of classes or interfaces that may not be present. Goal: minimal code changes for custom class loader authors Goal: If we need to make tradeoffs on how much thought and changes are needed for custom class loader authors, depending on whether they override findClass(String) or loadClass(String)/loadClass(String, boolean), make it simpler for those who override findClass(String). There are more of those custom class loaders and generally to override either loadClass(...) already requires dealing with more class loader complexity. Class loader Deadlock Problem Brief Overview of Class Loading Interactions Between Class loader and the VM Class loading requires cooperation between the VM and user level class loaders. Specifically, when the VM is performing constant pool resolution, among other things, the VM needs to call out to the user level class loader in order to find and define the requested class. On the other side, the class loader responsible for loading a class needs to call into the VM to determine if the class has already been loaded (findLoadedClass), and to define the class based on a bytecode stream (defineClass). The current recommended logic in detail is: User Level Class Loader VM constant pool resolution first acquire class loader lock -->private synchronized loadClassInternal(String) <--- calls out to loadClassInternal(String) | public loadClass(String) | protected synchronized loadClass(String,boolean) | protected final findLoadedClass(...) ---> VM SystemDictionary cache lookup | (can not trigger further class loading) | delegate (e.g. parent.loadClass(String)) | protected findClass(String) | reads in bytes | protected final defineClass(...) ---> resolves superclasses and superinterfaces | which recursively calls out to -------------------------------------------------- <--- loadClassInternal(String) Custom class loader authors are encouraged to override findClass(String) to allow them to determine where or how the byte code stream is obtained. Some custom class loader authors override loadClass (String) or loadClass(String, boolean) to be able to change the delegation strategy. Currently many class loading interactions are synchronized on the class loader lock. This works well for class loader delegation that assumes a strictly tree based delegation hierarchy. Customers have requested the ability to delegate to arbitrary class loaders. Currently this can cause deadlocks if class loaders delegate to each other without a fixed ordering. Sample Deadlock Scenario: non-tree based delegation hierarchy Class Hierarchy: class A extends B class C extends D ClassLoader Delegation Hierarchy: Custom Classloader CL1: directly loads class A delegates to custom ClassLoader CL2 for class B Custom Classloader CL2: directly loads class C delegates to custom ClassLoader CL1 for class D Thread 1: Use CL1 to load class A (locks CL1) defineClass A triggers loadClass B (try to lock CL2) Thread 2: Use CL2 to load class C (locks CL2) defineClass C triggers loadClass D (try to lock CL1) Proposed Solution User Level Class Loader VM constant pool resolution If ParallelCapable:lock class/class loader pair else acquire class loader lock *DEPRECATE*: private synchronized loadClassInternal |-->public loadClass(String) <--- calls out to loadClass(String) | call loadClass(String,boolean) (*no longer synchronized*) | *if ParallelCapable:* | *synchronize on a class-name-based-lock* | *else synchronize on "this" (backward compatibility)* | protected final findLoadedClass(...) ---> VM SystemDictionary cache lookup | (can not trigger further class loading) | delegate (e.g. parent.loadClass(String)) | protected findClass(String) | reads in by | protected final defineClass(...) ---> resolves superclasses and superinterfaces | which recursively calls out to -------------------------------------------------- <--- loadClass(String) API Modifications Java APIs New protected static method in java.lang.ClassLoader, boolean registerAsParallelCapable() To be called once from class initialization of a subclass of java.lang.ClassLoader which supports parallel loading of classes. In order for a a class loader class to register itself as parallel capable, all of its superclasses which are instances of java.lang.ClassLoader must also already be registered as parallel capable when invoking this protected static method. Otherwise the request will be ignored. registerAsParallelCapable() will return true if the class loader successfully registers as parallelCapable, even if accidentally called a second time. It will return false on failure. java.lang.ClassLoader API changes: Deprecate private loadClassInternal(), retain temporarily for backward compatibility Remove the synchronized keyword from: protected synchronized Class<?> loadClass(String name, boolean resolve) method. public synchronized void setDefaultAssertionStatus(boolean enabled) public synchronized void setPackageAssertionStatus(String packageName, boolean enabled) public synchronized void setClassAssertionStatus(String className, boolean enabled) public synchronized void clearAssertionStatus() VM Flags for early access testing AlwaysLockClassLoader - default false. Provided for ParallelClassLoader that has a bug in handling multiple threads in parallel. Expected to ship in product. AllowParallelDefineClass - default false. If two threads try to define the same class/class loader pair in parallel, throw linkageError, duplicate class definition. If you change the flag, we would allow parallel defineClass requests, using the result of the first requester. This would require a JVMS clarification. Experimental in product for early access feedback only, not expected to ship. MustCallLoadClassInternal - default false. In case a customer actually depended on this call, perhaps expecting to find this on the stack. Note that this will only enforce calling loadClassInternal(String). For instances of parallel capable class loaders, neither the VM nor loadClassInternal(String) will acquire the class loader lock. Expected to ship in product for one or two releases until deprecated loadClassInternal(String) can be safely removed. Java <-> VM interface changes VM calls loadClass(String) NOT loadClassInternal(String) always (note: backward compatibility -XX:+MustCallLoadClassInternal flag override) VM acquires class loader lock only if class loader instance is not a ParallelCapable class java.lang.ClassLoader's instance constructor will look up the class for a new class loader, and if it is registered as parallel capable, set a private instance field, parallelLockMap to non-null for the vm to query. Class Loader changes required java.lang.ClassLoader: Invoke registerAsParallelCapable() during class initialization Modify the code to ensure thread-safe concurrency Ensure no internal methods synchronize on the class loader object for parallel capable class loaders Ensure all critical sections are safe when executed by multiple threads loading different classes protected loadClass(String, boolean) changes: Remove synchronized keyword If "this" is not a parallel capable class loader, synchronize on "this" for backward compatibility else synchronize on a class-name-based-lock The synchronization in protected loadClass(String, boolean) ensures that defineClass(...) will not be called multiple times in parallel for the same class name/class loader pair. AssertionStatus related APIs Replace synchronized keyword with internal synchronization logic for: public synchronized void setDefaultAssertionStatus(boolean enabled) public synchronized void setPackageAssertionStatus(String packageName, boolean enabled) public synchronized void setClassAssertionStatus(String className, boolean enabled) public synchronized void clearAssertionStatus() Internal synchronization logic: If "this" is not a ParallelCapable class loader, synchronize on "this" for backward compatibility else synchronize on a dedicated internal lock object for all assertion related fields add new protected static method: registerAsParallelCapable() If all superclasses which are instances of java.lang.ClassLoader succeeded in registering as ParallelCapable, the calling class will also successfully register as parallel capable. java.security.SecureClassLoader Invoke registerAsParallelCapable() during class initialization Modify the code to ensure thread-safe concurrency Actually, no modifications were necessary java.net.URLClassLoader: Invoke registerAsParallelCapable() during class initialization Modify the code to ensure thread-safe concurrency details: TBD Suggested Model for Custom Class Loaders Custom class loaders that have no history of deadlocks require no changes. Custom class loaders that support a non-hierarchical delegation model and so are candidates for deadlock may choose to adopt the new mechanism. If you adopt this new mechanism, you need to modify all custom class loaders that could interact in a deadlock. Current class loaders frequently rely on the synchronization on the class loader lock provided by the enclosing methods loadClassInternal and protected loadClass(String, boolean). To resolve the current deadlocking on class loader locks, finer grained locking is needed. For class loader classes that successfully register themselves as parallel capable, the java.lang.ClassLoader class will no longer synchronize on the current class loader object when called to load a class. Instead java.lang.ClassLoader uses its own private lock which is unique for each class name. This allows concurrent loading of different classes for the same class loader instance. This locking logic is in the protected loadClass(String, boolean) method and will be passed to custom parallel class loaders through inheritance if not overridden. Recommended modifications for custom class loaders that need to avoid deadlocks: 1. REQUIRED: In the class initializer, invoke java.lang.ClassLoader registerAsParallelCapable() This registration indicates that all instances of this class loader class are multi-thread safe for concurrent class loading If the registration succeeds, java.lang.ClassLoader will allow concurrent loading of classes for the same class loader instance 2. REQUIRED: Modify the code to ensure multi-thread safety Decide upon an internal locking scheme. E.g. java.lang.ClassLoader uses a class-name-based locking scheme. Remove all synchronization on the class loader lock Ensure that critical sections are safe for multiple threads loading different classes. 3. REQUIRED: Ensure that all class loader classes that this custom class loader extends also invoke registerAsParallelCapable() in the class initializer and ensure they are multi-thread safe for concurrent class loading 4.a Class loaders that invoke registerAsParallelCapable(), that override findClass(String), (recommended overriding): No further modifications required 4.b. Class loaders that invoke registerAsParallelCapable(), that override protected loadClass(String, boolean) or public loadClass(String) We recommend that you override findClass(String) instead, if at all possible To ensure that the protected defineClass(...) method is only called once for the same class name, you need to implement a finer-grained locking scheme. One option would be to adopt the class name based locking mechanism from java.lang.ClassLoader's protected loadClass(String, boolean) method. Internal Implementation Details The most important point to make is that all class loaders that want the deadlock fix, must be cleaned up to ensure that they are multi-thread safe, i.e. that they allow the class loader to load multiple classes at the same time. The basic approach is to remove synchronization on the class loader itself, and provide smaller granularity locking for critical sections. It is critical that there be no synchronization on the class loader lock for parallel capable class loaders. Given that class loading is frequently triggered implicitly, e.g. by newInstance, and given the interactions between the VM and class loaders, acquiring the class loader lock holds the risk of causing a deadlock. All of the JRE class loaders which customers extend to create custom class loaders need to invoke registerAsParallelCapable(). They all must be made multi-thread safe for concurrent class loading of different class name/class loader pairs, so methods inherited by parallel capable custom class loaders are thread-safe. These include: java.lang.ClassLoader java.security.SecureClassLoader java.net.URLClassLoader javax.management.loading.MLet javax.management.loading.PrivateMlet All of the JRE class loaders which custom class loaders delegate to in the traditional tree-based hierarchy in theory would not require changes due to delegation. Given that they are supposed to only delegate further up the hierarchy, they should not participate in a deadlock. If at a future time any of these class loaders were to adopt an alternative delegation policy, that for instance would allow more direct delegation to a specific class loader, then it would be necessary to make these class loaders multi-thread safe. Bootstrap class loader Extension class loader: sun.misc.Launcher$ExtClassLoader System class loader sun.misc.launcher.AppClassLoader sun.Applet.AppletClassLoader Other class loaders, that are not extended and not delegated to by custom class loaders do not require any changes. These include: java.rmi.server.RMIClassLoader com.sun.jnlp.JNLPClassLoader All sun.* packages are private and customers can not extend sun.plugin._javascript_.JSClassLoader sun.plugin.security.PluginClassLoader sun.plugin2.applet.JNLP2ClassLoader sun.plugin2.applet.Applet2ClassLoader sun.plugin2.applet.Plugin2ClassLoader Assumptions: The DownloadManager does not load classes, it copies down files and the bootstrap class loader retries. VM Behavior Changes Part of the Sincere apologies on how long it has taken us to fix this problem. Part of the reason this fix has taken so long is that we first needed to modify the VM in the following ways: Fixed problems with circularity detection (JDK5) Modified the bootstrap class loader to be a parallel class loader (JDK6) Modified the VM common class resolution logic to handle parallel class loading (HotSpot 10) Details of VM handling of different cl Details of VM handling of specific class loader cases: Traditional class loaders JDK 6: lock class loader object lock, call loadClassInternal(String) New: when using traditional class loaders to load classes, lock class loader object lock, call loadClass(String) -XX:+UnlockDiagnosticOptions -XX:+UnsyncloadClass - DEPRECATED JDK 6: no class loader object lock for class loaders, call loadClass(String) allow parallel overall class resolution, parallel superclass loading, and parallel defineClass() calls parallel defineClass() handling: if there are two define class requests at the same time for the same class/class loader pair, the second requestor waits for the first requestor and returns the first requestor's results, instead of throwing a linkageError. New: support for flags is the same as in JDK6 parallelCapable JDK 6: same behavior as traditional class loaders New: no class loader object lock for class loaders, call loadClass(String) allow parallel overall class resolution, and parallel superclass loading parallel defineClass requests will throw LinkageErrors (for early access experimentation: see AllowParallelDefineClass flag) bootstrap class loader JDK6: no global lock, allow parallel overall class resolution wait for parallel superclass loads to complete parallel defineClass requests will throw LinkageErrors New: same as JDK6 breaking the class loader lock JDK6: special handling: for both overall class resolution and superclass loads: first requestor completes, others wait for completion New: same as JDK6 Alternatives Considered and Not Chosen Marker interface. Proposal: Provide a marker interface which a class loader can implement to declare that it is a parallel capable class loader. Reasons not chosen: It is key that a custom class loader not inherit registration as a parallel capable class loader. For class loaders to be able to run both with this new mechanism and on older JREs, we need a mechanism that has a way to dynamically check if the new mechanism is in place. With a new API, the class loader can check for the existence of the new method to determine whether the JRE supports parallel classloading. We can't extend an interface or subclass a new class which did not exist on an older version. Annotations Proposal: Use annotations to indicate that a class loader is parallelCapable. Reason not chosen: Annotations were explicitly designed to NOT modify VM runtime behavior, only to provide information, i.e. annotations were not intended to be a macro language for the VM. If run on a VM that does not understand an annotation, the behavior should not change. new loadClassXXX(String) API instead of registration Proposal: Add a new protected method loadClass2(String). The VM calls loadClass2(String), which acquires the class loader lock and then calls loadClass(String). Class loaders that support parallel loading override loadClass2(String) so that it takes a per-class-name lock instead and then calls loadClass(String). This also requires removing the synchronized keyword from loadClass(String, boolean). Reasons not chosen: The VM would have not way of knowing if the class loader supports parallel class loading or not. For prevent deadlock the VM needs to allow parallel class resolution and parallel superclass loading. For backward compatibility, the VM needs to not allow parallel class resolution and parallel superclass loading. There are multiple paths into class loading today which must continue to work for backward compatibility. Custom class loaders that call loadClass(String, boolean) must continue to work, so for them, loadClass(String, boolean) needs to acquire the class loader lock if they are calling an existing class loader, or not acquire the lock if they are calling a parallel capable class loader. new loadClassXXX(String) API in addition to registration mechanism Proposal: In addition to a registration mechanism, introduce a new protected loadClassUnsync(String, boolean) that is not synchronized. Change protected synchronized loadClass(String, boolean) to simply call loadClassUnsync(String, boolean). VM calls loadClass(String) which dispatches to loadClass(String, boolean) or loadClassUnsync(String, boolean) depending on whether the class loader is parallel capable or not. Reasons not chosen: Strongly considered and in fact prototyped. While we believe this could work technically, we believe the current proposal is the simplest we have so far devised in terms of requiring minimal changes for custom class loader authors who need to deal with deadlocks, and no changes for others. Requires both the same changes the current proposal does for custom class loader author and additional changes: All custom class loader authors would need to provide their own synchronization mechanism, i.e. we would not provide the per-class-name synchronization that we do today Custom class loader authors that currently override findClass(String) would now need to provide loadClassUnsync(String, boolean), i.e. essentially needing to override loadClass(String, boolean) functionality. Our goal is to encourage custom class loader authors, particularly those that just want to modify where classes are found, to only have to override findClass(String). Per-instance registerAsParallelCapable(): Proposal: Instead of registering a class loader class as parallel capable, register a single class loader instance. Reason not chosen. Not needed. The registration is intended to inform that a class loader is multi-thread safe. That information is meaningful on a per-class bases. If a custom class loader author wishes, they can go back to locking on the classloader object for instances in which they would like to disable the parallel class loading capability in their implementation. By registering as parallelCapable, they have that option. Tracking of parallel capable is based on the Class instance of the class loader which registered as parallel capable and not based on the class name. So if multiple custom loaders choose the same class name, or if someone chooses to load the same class loader classfile with different class loaders, those will be tracked independently. New LinkageError for parallel duplicate requests Proposal: It has been suggested that one way of handing parallel duplicate requests would be to define a new LinkageError subclass passing back the original defined class, rather than the current error of "duplicate". I think the intention of this proposal was that the caller of defineClass() could check if the original class matched the requested class, and if so, do a findLoadedClass() to use the previously defined class. Reason not chosen: A problem with the proposal is that the VM does not actually cache the original unprocessed bytestream, and it would not be worth the overhead to do so. Ability to override the requirement that all superclasses must be parallelCapable. Proposal: Some custom class loader authors would like to state that a given class loader is parallelCapable and they guarantee that all superclasses are parallelCapable even though they are not registered as such. Reasons not chosen: A strong concern is that authors of the superclass would not know that future changes would be expected to support the multi-thread safety that parallelCapable implies, and so there would be a strong risk of breaking undocumented, unofficial, backward compatibility. Open Issues Use the term Async instead of parallelCapable Naming suggestions are welcome Modify JVMS and defineClass(...) behavior Proposal: There is a suggestion to modify defineClass() behavior such that for parallel capable class loaders, if two defineClass(...) requests for the same class/class loader pair occur in parallel, rather than throwing a LinkageError for a duplicate definition, defineClass() would have enforce first-definition-wins-behavior. All defineClass(...) requests would parse the byte stream for ClassFileFormatError. The first requester will complete the definition process, subsequent requestors will wait and use the results of the first request. The prototype will offer a flag, -XX:+AllowParallelDefineClass so that you can provide feedback based on experimentation. Protected method for subclass re-use for the default locking model Proposal: Provide a method in java.lang.ClassLoader that would allow subclasses to re-use the default per-class-name locking model, making modifications easier for custom loaders that override loadClass(...). Current prototype: Object getClassLoadingLock(String className) which will either return the per-class-name lock for that class loader, or the class loader object for non-parallel capable class loaders. This allows the caller to just synchronize on the result without having to know whether the current class loader is parallel capable or not. Another less common scenario for use would be if a custom non-parallel capable class loader extends a parallel capable class loader. If the parallel capable class loader invokes loadCass(...) on the non-parallel subclass, and the subclass does not override loadClass(...), the common loadClass(...) logic may wish to synchronize on the appropriate lock depending on whether the current class loader is parallel capable or not. Please let us know if you need this API. Given the goal of simplicity, we will only add APIS that are deemed must-haves. Also, by default we would mark this API final. If you need this to be able to override this API let us know.