The problem: Stale references allowed and noted in comments:
java.security.Permissions java.security.BasicPermissions.BasicPermissionCollection The stale reference in Permissions is an AllPermission object - an optimisation. If a thread doesn't see the current value, it just checks the internal Map, which is synchronised, no biggy. Problem is, Permissions is a heterogenous PermissionCollection, it contains a Map, synchronzed thread access, which prevents a similar optimisation in the homogenous BasicPermissionCollection from being seen in the stale state. Every ProtectionDomain has its own Permissions and each Permission class type has it's own unique PermissionCollection shared with all others with the same type for a ProtectionDomain. I replaced Permissions with a class called ConcurrentPermissions that uses a ConcurrentMap Trouble is BasicPermissionCollection is no longer protected by synchronization in Permissions. BasicPermissionCollection now exposed to multiple threads has a stale reference optimisation for wildcard * permissions. What happens in my concurrent policy implementation is the Permission isn't necessarily found in the BasicPermissionCollection by a second thread, so it checks the PermissionGrants (immutable objects that contain data from policy files or dynamic grants) again and adds all the permissions to BasicPermissionCollection again. So it doesn't fail, but it doesn't scale well with contention, because you've still got the synchronisation bottleneck, can't see the Permission and have to process again, wasting resources, on the second occassion. Problem is, BasicPermissionCollection is the bread and butter PermissionCollection implementation many Permission classes use. Now you have to remember, these classes were designed well before concurrency was ever a consideration. Nowadays these classes would be immutable, since policy's don't change much, they're mostly read access. But I can't change it because many are part of the decision process. Now I could put a synchronized wrapper PermissionCollection class around these things, which fixes the bug, creating long lived objects that live on the heap and will likely cause L2 cache misses or contended locks. How about something different? Create the PermissionCollection's on demand, then discard immediately after use. The Permission objects themselves are long lived immutable objects. Why? It'll be used only by one thread, so the jvm will optimise out the synchronised locks. The object will be created on the threads local memory stack, instead of the heap and die in the young generation, so it doesn't incur gc heap generation movements or memory heap copy to cpu cache stalls. But what about single thread applications or those with few threads and little contention? They would run slower, although object allocation costs aren't as bad as people think, say 10 to 20 cpu cycles compared to 200 for a cache miss, or worse for a contended lock. Pattern matching of strings is the most expensive computation of most permission decisions and has to be repeated for every ProtectionDomain on the call stack for each thread, the impact on single core machines won't be much. I can test for that, but not the high end stuff. Arrghh decisions! Not enough test hardware. Cheers, Peter.