[
https://issues.apache.org/jira/browse/NETBEANS-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449908#comment-16449908
]
Jean-Marc Borer edited comment on NETBEANS-58 at 4/24/18 2:27 PM:
------------------------------------------------------------------
@Geertjan, no it did not happen before. Now, I don't know from when exactly it
started, I mean which JDK/NB version combination. Actually it doesn't matter.
Now it no longer works and the reason has been quite clearly identified: dead
locking while querying the Keyring.
If you look at the extract from my block IDE thread dump you will see that:
1) "Thread-8" takes the lock {color:#ff0000}0x00000000c0d64370 {color}in
sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:200)
Then further down the call stack, it creates a task that will wait forever
reading the a value from Keyring at
org.netbeans.api.keyring.Keyring.read(Keyring.java:144)
2) This task is posted on thraed "*org.netbeans.api.keyring.Keyring*" where it
tries to get the lock on {color:#ff0000}0x00000000c0d64370{color}. Boom
deadlock...
This blocks all threads on the ModuleManager$SystemClassLoader lock
{color:#ff0000}0x00000000c0d64370 {color}for every other thread including the
AWT Event one which leads to the HMI freeze.
{noformat}
"Thread-8" #49 prio=6 os_prio=0 tid=0x00000000256a5000 nid=0xdb8 waiting on
condition [0x000000002cd7a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f05d6c10> (a
org.openide.util.RequestProcessor$RPFutureTask)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.netbeans.api.keyring.Keyring.read(Keyring.java:144)
at
org.netbeans.core.ProxySettings.getAuthenticationPassword(ProxySettings.java:230)
at
org.netbeans.core.NbAuthenticator.getPasswordAuthentication(NbAuthenticator.java:87)
at
java.net.Authenticator.requestPasswordAuthentication(Authenticator.java:317)
- locked <0x00000000c2c66008> (a org.netbeans.core.NbAuthenticator)
at
sun.net.www.protocol.http.spnego.NegotiateCallbackHandler.getAnswer(NegotiateCallbackHandler.java:65)
at
sun.net.www.protocol.http.spnego.NegotiateCallbackHandler.handle(NegotiateCallbackHandler.java:86)
at
com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:858)
at
com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:704)
at
com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at sun.security.jgss.GSSUtil.login(GSSUtil.java:258)
at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
at
sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
at
sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
sun.security.jgss.spnego.SpNegoContext.GSS_initSecContext(SpNegoContext.java:882)
at
sun.security.jgss.spnego.SpNegoContext.initSecContext(SpNegoContext.java:317)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
sun.net.www.protocol.http.spnego.NegotiatorImpl.init(NegotiatorImpl.java:108)
at
sun.net.www.protocol.http.spnego.NegotiatorImpl.<init>(NegotiatorImpl.java:117)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
sun.net.www.protocol.http.Negotiator.getNegotiator(Negotiator.java:63)
at
sun.net.www.protocol.http.NegotiateAuthentication.isSupportedImpl(NegotiateAuthentication.java:130)
- locked <0x00000000f0e74ed8> (a java.lang.Class for
sun.net.www.protocol.http.NegotiateAuthentication)
at
sun.net.www.protocol.http.NegotiateAuthentication.isSupported(NegotiateAuthentication.java:102)
- locked <0x00000000c0d64370> (a
org.netbeans.ModuleManager$SystemClassLoader)
at
sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:200)
at
sun.net.www.protocol.http.AuthenticationHeader.<init>(AuthenticationHeader.java:143)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1626)
- locked <0x00000000f0e75008> (a
sun.net.www.protocol.http.HttpURLConnection)
at
sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91)
at
sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466)
at
sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464)
at java.security.AccessController.doPrivileged(Native Method)
at
java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463)
- locked <0x00000000f0e75008> (a
sun.net.www.protocol.http.HttpURLConnection)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at
org.netbeans.modules.welcome.content.RSSFeed.findInputSource(RSSFeed.java:233)
at
org.netbeans.modules.welcome.content.RSSFeed.buildItemList(RSSFeed.java:196)
at
org.netbeans.modules.welcome.content.RSSFeed$Reload.run(RSSFeed.java:308)
"org.netbeans.api.keyring.Keyring" #26 daemon prio=1 os_prio=-2
tid=0x0000000019593000 nid=0x1640 waiting for monitor entry [0x000000002655e000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.netbeans.ModuleManager$SystemClassLoader.getResourcesImpl(ModuleManager.java:708)
- waiting to lock <0x00000000c0d64370> (a
org.netbeans.ModuleManager$SystemClassLoader)
at org.netbeans.ProxyClassLoader.getResources(ProxyClassLoader.java:390)
at
org.openide.util.lookup.MetaInfServicesLookup.search(MetaInfServicesLookup.java:205)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookup(MetaInfServicesLookup.java:156)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookupResult(MetaInfServicesLookup.java:135)
at
org.openide.util.lookup.AbstractLookup.lookup(AbstractLookup.java:483)
at
org.openide.util.lookup.ProxyLookup$R.initResults(ProxyLookup.java:390)
at
org.openide.util.lookup.ProxyLookup$R.myBeforeLookup(ProxyLookup.java:673)
at
org.openide.util.lookup.ProxyLookup$R.computeResult(ProxyLookup.java:553)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:513)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:509)
at org.openide.util.Lookup.lookupAll(Lookup.java:312)
at org.netbeans.api.keyring.Keyring.provider(Keyring.java:89)
- locked <0x00000000f0525f68> (a java.lang.Class for
org.netbeans.api.keyring.Keyring)
at org.netbeans.api.keyring.Keyring.readImpl(Keyring.java:105)
- locked <0x00000000f0525f68> (a java.lang.Class for
org.netbeans.api.keyring.Keyring)
at org.netbeans.api.keyring.Keyring.access$100(Keyring.java:75)
at org.netbeans.api.keyring.Keyring$1.call(Keyring.java:128)
at org.netbeans.api.keyring.Keyring$1.call(Keyring.java:125)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
at
org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
at
org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
{noformat}
The authenticator plugin did not help because as far as I understand it
improves SPNEGO support not the way the Keyring is invoked. I am sitting behind
a NTLMv2 proxy. Authentication works (and worked) but it dead locks...
If you look at the Keyring class code you see:
{code:java}
public static char[] read(@NonNull final String key) {
Parameters.notNull("key", key);
try {
final Future<char[]> result = KEYRING_ACCESS.submit(new Callable<char[]>() {
@Override
public char[] call() throws Exception {
return Keyring.readImpl(key);
}
});
if (SwingUtilities.isEventDispatchThread()) {
if (!result.isDone()) {
try {
// lets wait in awt to avoid flashing dialogs
return result.get(SAFE_DELAY, TimeUnit.MILLISECONDS);
} catch (TimeoutException ex) {
// show progress dialog
return ProgressUtils.showProgressDialogAndRun(
new ProgressRunnable<char[]>(result), Bundle.MSG_KeyringAccess(), false);
}
}
}
return result.get();
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
} catch (ExecutionException ex) {
LOG.log(Level.INFO, null, ex);
}
return null;
}
{code}
The intention is to get the keyring value off the EDT thread. However the
problem is the the the retrieval thread will lock on
{color:#ff0000}0x00000000c0d64370{color} which in turn locks the Thread-8 for
ever and therefore lock {color:#ff0000}0x00000000c0d64370{color} will never be
released. Unfortunately EDT is also relying to the lock
{color:#ff0000}0x00000000c0d64370{color}:
{noformat}
"AWT-EventQueue-0" #22 prio=6 os_prio=0 tid=0x000000001d16f800 nid=0x1950
waiting for monitor entry [0x000000001f35b000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.netbeans.ModuleManager$SystemClassLoader.getResourcesImpl(ModuleManager.java:708)
- waiting to lock <0x00000000c0d64370> (a
org.netbeans.ModuleManager$SystemClassLoader)
at org.netbeans.ProxyClassLoader.getResources(ProxyClassLoader.java:390)
at
org.openide.util.lookup.MetaInfServicesLookup.search(MetaInfServicesLookup.java:205)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookup(MetaInfServicesLookup.java:156)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookupResult(MetaInfServicesLookup.java:135)
at
org.openide.util.lookup.AbstractLookup.lookup(AbstractLookup.java:483)
at
org.openide.util.lookup.ProxyLookup$R.initResults(ProxyLookup.java:390)
at
org.openide.util.lookup.ProxyLookup$R.myBeforeLookup(ProxyLookup.java:673)
at
org.openide.util.lookup.ProxyLookup$R.computeResult(ProxyLookup.java:553)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:513)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:509)
at
org.netbeans.modules.versioning.core.VersioningManager.init(VersioningManager.java:224)
- locked <0x00000000f0e81540> (a java.util.ArrayList)
at
org.netbeans.modules.versioning.core.VersioningManager.getInstance(VersioningManager.java:118)
- locked <0x00000000f0e815b0> (a java.lang.Class for
org.netbeans.modules.versioning.core.VersioningManager)
at
org.netbeans.modules.versioning.core.filesystems.VCSFilesystemInterceptor.<clinit>(VCSFilesystemInterceptor.java:65)
at
org.netbeans.modules.versioning.masterfs.VersioningAnnotationProvider.annotateIcon(VersioningAnnotationProvider.java:71)
at
org.netbeans.modules.masterfs.ui.FileBasedFSWithUI$UiDecorator.annotateIcon(FileBasedFSWithUI.java:77)
at
org.openide.loaders.DataFolder$FolderNode.getIcon(DataFolder.java:1250)
at org.openide.nodes.FilterNode.getIcon(FilterNode.java:564)
at org.openide.nodes.FilterNode.getIcon(FilterNode.java:564)
at
org.netbeans.modules.project.ui.PhysicalView$ProjectIconNode.getIcon(PhysicalView.java:426)
at org.openide.nodes.FilterNode.getIcon(FilterNode.java:564)
at
org.netbeans.modules.project.ui.ProjectsRootNode$BadgingNode.getIcon(ProjectsRootNode.java:931)
at
org.netbeans.modules.project.ui.ProjectsRootNode$BadgingNode.getIcon(ProjectsRootNode.java:925)
at
org.openide.explorer.view.VisualizerNode.getIcon(VisualizerNode.java:558)
at
org.openide.explorer.view.NodeRenderer.configureFrom(NodeRenderer.java:238)
at
org.openide.explorer.view.NodeRenderer.getTreeCellRendererComponent(NodeRenderer.java:163)
at
javax.swing.plaf.basic.BasicTreeUI$NodeDimensionsHandler.getNodeDimensions(BasicTreeUI.java:2807)
at
javax.swing.tree.AbstractLayoutCache.getNodeDimensions(AbstractLayoutCache.java:492)
at
javax.swing.tree.FixedHeightLayoutCache.getBounds(FixedHeightLayoutCache.java:553)
at
javax.swing.tree.FixedHeightLayoutCache.getBounds(FixedHeightLayoutCache.java:199)
at
javax.swing.tree.AbstractLayoutCache.getPreferredWidth(AbstractLayoutCache.java:232)
at
javax.swing.plaf.basic.BasicTreeUI.updateCachedPreferredSize(BasicTreeUI.java:1897)
at
javax.swing.plaf.basic.BasicTreeUI.getPreferredSize(BasicTreeUI.java:2003)
at
javax.swing.plaf.basic.BasicTreeUI.getPreferredSize(BasicTreeUI.java:1991)
at javax.swing.JComponent.getPreferredSize(JComponent.java:1662)
at
javax.swing.ScrollPaneLayout.layoutContainer(ScrollPaneLayout.java:791)
at
org.openide.explorer.view.TreeView$ExplorerScrollPaneLayout.layoutContainer(TreeView.java:1709)
at java.awt.Container.layout(Container.java:1510)
at java.awt.Container.doLayout(Container.java:1499)
at java.awt.Container.validateTree(Container.java:1695)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validate(Container.java:1630)
- locked <0x00000000c04a13d8> (a java.awt.Component$AWTTreeLock)
at
org.netbeans.core.windows.view.ui.DesktopImpl$LayeredLayout.layoutContainer(DesktopImpl.java:554)
at java.awt.Container.layout(Container.java:1510)
at java.awt.Container.doLayout(Container.java:1499)
at java.awt.Container.validateTree(Container.java:1695)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validate(Container.java:1630)
- locked <0x00000000c04a13d8> (a java.awt.Component$AWTTreeLock)
at javax.swing.RepaintManager$3.run(RepaintManager.java:711)
at javax.swing.RepaintManager$3.run(RepaintManager.java:709)
at java.security.AccessController.doPrivileged(Native Method)
at
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at
javax.swing.RepaintManager.validateInvalidComponents(RepaintManager.java:708)
at
javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1731)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
at
org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
{noformat}
Maybe a solution would be not to request the Keyring value on a separate thread
(line org.netbeans.api.keyring.Keyring.read(Keyring.java:144))but do it
synchronously. As the call context already holds the
org.netbeans.ModuleManager$SystemClassLoader lock it would not block?
Here the full [^nb-freeze-dump.txt]
was (Author: jmborer):
@Geertjan, no it did not happen before. Now, I don't know from when exactly it
started, I mean which JDK/NB version combination. Actually it doesn't matter.
Now it no longer works and the reason has been quite clearly identified: dead
locking while querying the Keyring.
If you look at the extract from my block IDE thread dump you will see that:
1) "Thread-8" takes the lock {color:#ff0000}0x00000000c0d64370 {color}in
sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:200)
Then further down the call stack, it creates a task that will wait forever
reading the a value from Keyring at
org.netbeans.api.keyring.Keyring.read(Keyring.java:144)
2) This task is posted on thraed "*org.netbeans.api.keyring.Keyring*" where it
tries to get the lock on {color:#ff0000}0x00000000c0d64370{color}. Boom
deadlock...
This blocks all threads on the ModuleManager$SystemClassLoader lock
{color:#ff0000}0x00000000c0d64370 {color}for every other thread including the
AWT Event one which leads to the HMI freeze.
{noformat}
"Thread-8" #49 prio=6 os_prio=0 tid=0x00000000256a5000 nid=0xdb8 waiting on
condition [0x000000002cd7a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f05d6c10> (a
org.openide.util.RequestProcessor$RPFutureTask)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.netbeans.api.keyring.Keyring.read(Keyring.java:144)
at
org.netbeans.core.ProxySettings.getAuthenticationPassword(ProxySettings.java:230)
at
org.netbeans.core.NbAuthenticator.getPasswordAuthentication(NbAuthenticator.java:87)
at
java.net.Authenticator.requestPasswordAuthentication(Authenticator.java:317)
- locked <0x00000000c2c66008> (a org.netbeans.core.NbAuthenticator)
at
sun.net.www.protocol.http.spnego.NegotiateCallbackHandler.getAnswer(NegotiateCallbackHandler.java:65)
at
sun.net.www.protocol.http.spnego.NegotiateCallbackHandler.handle(NegotiateCallbackHandler.java:86)
at
com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:858)
at
com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:704)
at
com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at sun.security.jgss.GSSUtil.login(GSSUtil.java:258)
at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
at
sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
at
sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
sun.security.jgss.spnego.SpNegoContext.GSS_initSecContext(SpNegoContext.java:882)
at
sun.security.jgss.spnego.SpNegoContext.initSecContext(SpNegoContext.java:317)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
sun.net.www.protocol.http.spnego.NegotiatorImpl.init(NegotiatorImpl.java:108)
at
sun.net.www.protocol.http.spnego.NegotiatorImpl.<init>(NegotiatorImpl.java:117)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
sun.net.www.protocol.http.Negotiator.getNegotiator(Negotiator.java:63)
at
sun.net.www.protocol.http.NegotiateAuthentication.isSupportedImpl(NegotiateAuthentication.java:130)
- locked <0x00000000f0e74ed8> (a java.lang.Class for
sun.net.www.protocol.http.NegotiateAuthentication)
at
sun.net.www.protocol.http.NegotiateAuthentication.isSupported(NegotiateAuthentication.java:102)
- locked <0x00000000c0d64370> (a
org.netbeans.ModuleManager$SystemClassLoader)
at
sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:200)
at
sun.net.www.protocol.http.AuthenticationHeader.<init>(AuthenticationHeader.java:143)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1626)
- locked <0x00000000f0e75008> (a
sun.net.www.protocol.http.HttpURLConnection)
at
sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91)
at
sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466)
at
sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464)
at java.security.AccessController.doPrivileged(Native Method)
at
java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463)
- locked <0x00000000f0e75008> (a
sun.net.www.protocol.http.HttpURLConnection)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at
org.netbeans.modules.welcome.content.RSSFeed.findInputSource(RSSFeed.java:233)
at
org.netbeans.modules.welcome.content.RSSFeed.buildItemList(RSSFeed.java:196)
at
org.netbeans.modules.welcome.content.RSSFeed$Reload.run(RSSFeed.java:308)
"org.netbeans.api.keyring.Keyring" #26 daemon prio=1 os_prio=-2
tid=0x0000000019593000 nid=0x1640 waiting for monitor entry [0x000000002655e000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.netbeans.ModuleManager$SystemClassLoader.getResourcesImpl(ModuleManager.java:708)
- waiting to lock <0x00000000c0d64370> (a
org.netbeans.ModuleManager$SystemClassLoader)
at org.netbeans.ProxyClassLoader.getResources(ProxyClassLoader.java:390)
at
org.openide.util.lookup.MetaInfServicesLookup.search(MetaInfServicesLookup.java:205)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookup(MetaInfServicesLookup.java:156)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookupResult(MetaInfServicesLookup.java:135)
at
org.openide.util.lookup.AbstractLookup.lookup(AbstractLookup.java:483)
at
org.openide.util.lookup.ProxyLookup$R.initResults(ProxyLookup.java:390)
at
org.openide.util.lookup.ProxyLookup$R.myBeforeLookup(ProxyLookup.java:673)
at
org.openide.util.lookup.ProxyLookup$R.computeResult(ProxyLookup.java:553)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:513)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:509)
at org.openide.util.Lookup.lookupAll(Lookup.java:312)
at org.netbeans.api.keyring.Keyring.provider(Keyring.java:89)
- locked <0x00000000f0525f68> (a java.lang.Class for
org.netbeans.api.keyring.Keyring)
at org.netbeans.api.keyring.Keyring.readImpl(Keyring.java:105)
- locked <0x00000000f0525f68> (a java.lang.Class for
org.netbeans.api.keyring.Keyring)
at org.netbeans.api.keyring.Keyring.access$100(Keyring.java:75)
at org.netbeans.api.keyring.Keyring$1.call(Keyring.java:128)
at org.netbeans.api.keyring.Keyring$1.call(Keyring.java:125)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
at
org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
at
org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
{noformat}
The authenticator plugin did not help because as far as I understand it
improves SPNEGO support not the way the Keyring is invoked. I am sitting behind
a NTLMv2 proxy. Authentication works (and worked) but it dead locks...
If you look at the Keyring class code you see:
{code:java}
public static char[] read(@NonNull final String key) {
Parameters.notNull("key", key);
try {
final Future<char[]> result = KEYRING_ACCESS.submit(new Callable<char[]>() {
@Override
public char[] call() throws Exception {
return Keyring.readImpl(key);
}
});
if (SwingUtilities.isEventDispatchThread()) {
if (!result.isDone()) {
try {
// lets wait in awt to avoid flashing dialogs
return result.get(SAFE_DELAY, TimeUnit.MILLISECONDS);
} catch (TimeoutException ex) {
// show progress dialog
return ProgressUtils.showProgressDialogAndRun(
new ProgressRunnable<char[]>(result), Bundle.MSG_KeyringAccess(), false);
}
}
}
return result.get();
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
} catch (ExecutionException ex) {
LOG.log(Level.INFO, null, ex);
}
return null;
}
{code}
The intention is to get the keyring value off the EDT thread. However the
problem is the the the retrieval thread will lock on
{color:#FF0000}0x00000000c0d64370{color} which in turn locks the Thread-8 for
ever and therefore lock {color:#FF0000}0x00000000c0d64370{color} will never be
released. Unfortunately EDT is also relying to the lock
{color:#FF0000}0x00000000c0d64370{color}:
{noformat}
"AWT-EventQueue-0" #22 prio=6 os_prio=0 tid=0x000000001d16f800 nid=0x1950
waiting for monitor entry [0x000000001f35b000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.netbeans.ModuleManager$SystemClassLoader.getResourcesImpl(ModuleManager.java:708)
- waiting to lock <0x00000000c0d64370> (a
org.netbeans.ModuleManager$SystemClassLoader)
at org.netbeans.ProxyClassLoader.getResources(ProxyClassLoader.java:390)
at
org.openide.util.lookup.MetaInfServicesLookup.search(MetaInfServicesLookup.java:205)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookup(MetaInfServicesLookup.java:156)
at
org.openide.util.lookup.MetaInfServicesLookup.beforeLookupResult(MetaInfServicesLookup.java:135)
at
org.openide.util.lookup.AbstractLookup.lookup(AbstractLookup.java:483)
at
org.openide.util.lookup.ProxyLookup$R.initResults(ProxyLookup.java:390)
at
org.openide.util.lookup.ProxyLookup$R.myBeforeLookup(ProxyLookup.java:673)
at
org.openide.util.lookup.ProxyLookup$R.computeResult(ProxyLookup.java:553)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:513)
at
org.openide.util.lookup.ProxyLookup$R.allInstances(ProxyLookup.java:509)
at
org.netbeans.modules.versioning.core.VersioningManager.init(VersioningManager.java:224)
- locked <0x00000000f0e81540> (a java.util.ArrayList)
at
org.netbeans.modules.versioning.core.VersioningManager.getInstance(VersioningManager.java:118)
- locked <0x00000000f0e815b0> (a java.lang.Class for
org.netbeans.modules.versioning.core.VersioningManager)
at
org.netbeans.modules.versioning.core.filesystems.VCSFilesystemInterceptor.<clinit>(VCSFilesystemInterceptor.java:65)
at
org.netbeans.modules.versioning.masterfs.VersioningAnnotationProvider.annotateIcon(VersioningAnnotationProvider.java:71)
at
org.netbeans.modules.masterfs.ui.FileBasedFSWithUI$UiDecorator.annotateIcon(FileBasedFSWithUI.java:77)
at
org.openide.loaders.DataFolder$FolderNode.getIcon(DataFolder.java:1250)
at org.openide.nodes.FilterNode.getIcon(FilterNode.java:564)
at org.openide.nodes.FilterNode.getIcon(FilterNode.java:564)
at
org.netbeans.modules.project.ui.PhysicalView$ProjectIconNode.getIcon(PhysicalView.java:426)
at org.openide.nodes.FilterNode.getIcon(FilterNode.java:564)
at
org.netbeans.modules.project.ui.ProjectsRootNode$BadgingNode.getIcon(ProjectsRootNode.java:931)
at
org.netbeans.modules.project.ui.ProjectsRootNode$BadgingNode.getIcon(ProjectsRootNode.java:925)
at
org.openide.explorer.view.VisualizerNode.getIcon(VisualizerNode.java:558)
at
org.openide.explorer.view.NodeRenderer.configureFrom(NodeRenderer.java:238)
at
org.openide.explorer.view.NodeRenderer.getTreeCellRendererComponent(NodeRenderer.java:163)
at
javax.swing.plaf.basic.BasicTreeUI$NodeDimensionsHandler.getNodeDimensions(BasicTreeUI.java:2807)
at
javax.swing.tree.AbstractLayoutCache.getNodeDimensions(AbstractLayoutCache.java:492)
at
javax.swing.tree.FixedHeightLayoutCache.getBounds(FixedHeightLayoutCache.java:553)
at
javax.swing.tree.FixedHeightLayoutCache.getBounds(FixedHeightLayoutCache.java:199)
at
javax.swing.tree.AbstractLayoutCache.getPreferredWidth(AbstractLayoutCache.java:232)
at
javax.swing.plaf.basic.BasicTreeUI.updateCachedPreferredSize(BasicTreeUI.java:1897)
at
javax.swing.plaf.basic.BasicTreeUI.getPreferredSize(BasicTreeUI.java:2003)
at
javax.swing.plaf.basic.BasicTreeUI.getPreferredSize(BasicTreeUI.java:1991)
at javax.swing.JComponent.getPreferredSize(JComponent.java:1662)
at
javax.swing.ScrollPaneLayout.layoutContainer(ScrollPaneLayout.java:791)
at
org.openide.explorer.view.TreeView$ExplorerScrollPaneLayout.layoutContainer(TreeView.java:1709)
at java.awt.Container.layout(Container.java:1510)
at java.awt.Container.doLayout(Container.java:1499)
at java.awt.Container.validateTree(Container.java:1695)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validate(Container.java:1630)
- locked <0x00000000c04a13d8> (a java.awt.Component$AWTTreeLock)
at
org.netbeans.core.windows.view.ui.DesktopImpl$LayeredLayout.layoutContainer(DesktopImpl.java:554)
at java.awt.Container.layout(Container.java:1510)
at java.awt.Container.doLayout(Container.java:1499)
at java.awt.Container.validateTree(Container.java:1695)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validateTree(Container.java:1704)
at java.awt.Container.validate(Container.java:1630)
- locked <0x00000000c04a13d8> (a java.awt.Component$AWTTreeLock)
at javax.swing.RepaintManager$3.run(RepaintManager.java:711)
at javax.swing.RepaintManager$3.run(RepaintManager.java:709)
at java.security.AccessController.doPrivileged(Native Method)
at
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at
javax.swing.RepaintManager.validateInvalidComponents(RepaintManager.java:708)
at
javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1731)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at
java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
at
org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
{noformat}
Maybe a solution would be not to request the Keyring value on a separate thread
(line org.netbeans.api.keyring.Keyring.read(Keyring.java:144))but do it
synchronously. As the call context already holds the
org.netbeans.ModuleManager$SystemClassLoader lock it would not block?
Here the full dump
> NB IDE or NB Platform freeze on startup (proxy with Negotiate auth)
> -------------------------------------------------------------------
>
> Key: NETBEANS-58
> URL: https://issues.apache.org/jira/browse/NETBEANS-58
> Project: NetBeans
> Issue Type: Bug
> Components: platform - Proxy
> Affects Versions: 8.2, 9.0
> Environment: Primarily Windows.
> Reporter: phansson
> Priority: Critical
> Attachments: NETBEANS-58-workaround1.diff, nb-freeze-dump.txt,
> netbeans.txt
>
>
> When any network operation is performed, such as attempting to contact
> NetBeans Update Center, the application (IDE or Platform) may freeze. Users
> will typically experience this on startup. It was reported in old bug tracker
> as [bug 248308|https://netbeans.org/bugzilla/show_bug.cgi?id=248308].
> The problem arises because of the fix JDK folks applied as a consequence of
> the reported [JDK-8032832
> bug|https://bugs.openjdk.java.net/browse/JDK-8032832]. This fix wasn't very
> clever IMO: it puts a lock on the classloader, thus introducing a range of
> other problems, one of them being that NetNeans IDE or NetBeans Platform will
> likely freeze on startup when it attempts a network operation. The fact that
> their fix made things worse (while no doubt fixing the original issue) has
> been reported as
> [JDK-8068184|https://bugs.openjdk.java.net/browse/JDK-8068184].
> h3. WHEN DOES IT HAPPEN?
> As the lock is introduced for authentication of type 'Negotiate' it of course
> only happens if there's a network proxy on the path which uses this type of
> authentication. Also known as SPNEGO. This form of authentication is in my
> experience very common in corporate networks, in particular those that base
> themselves on the Microsoft stack. But a person on Oracle's own internal
> network, such as a JDK developer, is most likely not exposed to it. :-)
> There's another condition for it to happen: The JRE runtime must be unable to
> provide 'credentials' (a Kerberos token) to the network proxy on its own.
> SPNEGO is really designed to be seamless and promptless. Support for it was
> added in Java 6. But later on Microsoft tightened the desktop security around
> obtaining the so-called 'session token' and the JDK folks were never able to
> work around this (unlike the makers of Chrome, FF, Opera, etc). Therefore, in
> real-life, SPNEGO in the JRE on Windows is no longer promptless: it will be
> forced to ask the user for credentials, thus negating the idea of SPNEGO. It
> is the prompting which causes the freeze. SPNEGO on Mac OS X and Linux is
> most likely working just fine and the bug will never be experienced.
> h3. HOW DO I KNOW IF I'M AFFECTED BY EXACTLY THIS BUG?
> This bug in this ticket is characterized by the fact that you'll always be
> able to find the following in your thread dump:
> {noformat}
> at
> sun.net.www.protocol.http.NegotiateAuthentication.isSupported(NegotiateAuthentication.java:<lineno>)
> - locked <OBJECTID> (a org.netbeans.ModuleManager$SystemClassLoader)
> {noformat}
> Note that the [Ctrl-Break
> method|https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr019.html]
> of obtaining a thread dump is favoured over jstack and other methods.
> h3. WHY DOES IT HAPPEN?
> There will be a lock held on the classloader object when the JRE's registered
> Authenticator is invoked. If the Authenticator does work on another thread,
> that other thread has a need for some classloading and the current thread
> needs to wait for the result of that thread, then bum!, there's a deadlock
> between the two threads. This means the lock on the classloader will never be
> released and it will ultimately affect other threads, such as the AWT
> dispatch thread (aka Swing EDT) which will then also lock. Then you have what
> the user experiences as a freeze.
> The NB Platform's own Authenticator, {{NbAuthenticator}}, does exactly what I
> described and will thus be triggering the deadlock. More precisely it will
> happen when NbAuthenticator calls Keyring. Does this mean the NbAuthenticator
> does something wrong? No, of course it doesn't. The real problem is the lock
> on the classloader. It is actually virtually impossible to design an
> Authenticator which doesn't trigger this problem. You cannot predict when
> classloading is needed. In fact it is very likely to be needed when
> application is still not "warm", i.e. during startup.
> h3. WORKAROUNDS
> *#1*
> If on Windows: Setting the following registry key:
> {{HKLM\System\CurrentControlSet\Control\Lsa\Kerberos\Parameters\allowtgtsessionkey}}
> to {{true}} will allow the JRE to obtain the session key, thus the JDK's HTTP
> classes will have no need to invoke Authenticator, thus deadlock will not
> happen. But this registry key only has effect for users who are _not_ local
> admins and since it is a HKLM such user will need to ask his administrator to
> do this change. There's probably zero chance in a million that a corporate
> network administrator will allow this change. After all, Microsoft introduced
> the tightened security for a reason and the admin will rightly ask why only
> the JRE needs this and not Chrome, IE, FF, Opera and so on?
> *#2*
> Convice the JDK folks that they've made a mess of it with this lock. I've
> pursued this avenue too. I've done that on the JDK security-dev mailing list.
> I've pointed to similar bug tickets for Eclipse IDE and IDEA and I got the
> attention of Weijun Wang ("Max") of Oracle who promised he would look at it.
> But to be honest the JDK people have a lot of other things on their plate and
> a fix will take some time.
> UPDATE 29-OCT-2017: To be fair to the JDK folks to issue only occurs with
> NB's own classloaders. So the chance they'll fix it at the JDK end is
> probably slim.
> Link:
> http://mail.openjdk.java.net/pipermail/security-dev/2017-August/016267.html
> *#3*
> Re-design the Authenticator in the Platform. As I cannot change the code in
> NB itself, I've created a workaround as a plugin. The recipe is described in
> [Comment 44|https://netbeans.org/bugzilla/show_bug.cgi?id=248308#c44] on the
> original NB bug ticket. This is really a workaround, not a fix. It will
> simply give up on attempting to obtain credentials if it discovers that it is
> likely to be in this deadlock scenario. Thus it leaves the application with
> no outbound network connectivity but it is still better than the freeze. It
> also alerts the user to the situation using a bubble notification.
> Links:
> https://bitbucket.org/phansson/netbeansnetworkauthenticator
> https://bitbucket.org/phansson/netbeansnetworkauthenticator/wiki/JDK-8068184%20Workaround
> *#4*
> It may help to set proxy username/password _explicitly_ in the NB's Options
> panel. But this is not a solution I recommend. It means you must put your AD
> password into NetBeans IDE Options. It will of course be static so once your
> AD password changes then you must remember to change there as well. And for
> the user to configure this, it requires that he can actually start the IDE or
> Platform app in the first place ... which is often not the case because of
> the freeze.
> *#5*
> Use a JRE prior to 8u20 or prior to 7u76. To most people this workaround is
> unacceptable.
> h3. CONCLUSION
> IMHO #3 is the most attractive solution for now. It doesn't exclude the user
> still doing #1 or #4 to get full benefit. The real solution is of course #2.
> .. and the very best solution long term is if JDK would have same support for
> SPNEGO on Windows as does 'regular' applications such as Chrome, FF, Opera,
> IE, Edge, etc. Then we wouldn't have the problem in the first place. This has
> been discussed intensively over the last 7-8 years but there's a somewhat
> religious tug of war between Sun/Oracle and Microsoft on the matter. The
> problem can be solved if the JDK would base itself on the Win32 SSPI api in
> this area, rather than the long-time deprecated Win32
> LsaCallAuthenticationPackage API. We are getting slightly off topic here and
> not appropriate for a bug ticket. :-)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@netbeans.apache.org
For additional commands, e-mail: commits-h...@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
