[
https://issues.apache.org/jira/browse/GEODE-10081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hale Bales updated GEODE-10081:
-------------------------------
Description:
We are using gfsh with a script to automatically launch a locator and a server.
During the launch of the locator, the process gfsh just freezes randomly. The
locator process is forked, but gfsh remains frozen.
A jstack on the process shows that the process is stuck on an internal
NativeLibrary.load call. This one happens when gfsh try to attach to the
process : internally the jvm loads the libattach.so library, but the call never
return.
To be complete, Geode is here deployed inside a docker container, using ubi8 as
base image, java is jdk11.0.11 - we have tested with 11.0.13, same issue.
The stack is the following (full jstack dump attached)
{code:java}
"main" #1 prio=5 os_prio=0 cpu=2402.78ms elapsed=1924.45s allocated=236M
defined_classes=10346 tid=0x00007fc904024000 nid=0x16 runnable
[0x00007fc90a84a000]
java.lang.Thread.State: RUNNABLE
at java.lang.ClassLoader$NativeLibrary.load0([email protected]/Native
Method)
at java.lang.ClassLoader$NativeLibrary.load([email protected]/Unknown
Source)
at
java.lang.ClassLoader$NativeLibrary.loadLibrary([email protected]/Unknown
Source)
- locked <0x00000000823efe60> (a java.util.HashSet)
at java.lang.ClassLoader.loadLibrary0([email protected]/Unknown Source)
at java.lang.ClassLoader.loadLibrary([email protected]/Unknown Source)
at java.lang.Runtime.loadLibrary0([email protected]/Unknown Source)
- locked <0x00000000828224b8> (a java.lang.Runtime)
at java.lang.System.loadLibrary([email protected]/Unknown Source)
at sun.tools.attach.VirtualMachineImpl.<clinit>([email protected]/Unknown
Source)
at
sun.tools.attach.AttachProviderImpl.attachVirtualMachine([email protected]/Unknown
Source)
at com.sun.tools.attach.VirtualMachine.attach([email protected]/Unknown
Source)
at
org.apache.geode.internal.process.MBeanProcessController.getJMXServiceURL(MBeanProcessController.java:227)
at
org.apache.geode.internal.process.MBeanProcessController.connect(MBeanProcessController.java:192)
at
org.apache.geode.internal.process.MBeanProcessController.invokeOperationOnTargetMBean(MBeanProcessController.java:159)
at
org.apache.geode.internal.process.MBeanProcessController.status(MBeanProcessController.java:136)
at
org.apache.geode.internal.process.MBeanProcessController.status(MBeanProcessController.java:81)
at
org.apache.geode.internal.process.MBeanOrFileProcessController.status(MBeanOrFileProcessController.java:37)
at
org.apache.geode.distributed.LocatorLauncher.statusWithWorkingDirectory(LocatorLauncher.java:1012)
at
org.apache.geode.distributed.LocatorLauncher.status(LocatorLauncher.java:940)
at
org.apache.geode.distributed.LocatorLauncher$LocatorState.fromDirectory(LocatorLauncher.java:2077)
at
org.apache.geode.management.internal.cli.commands.StartLocatorCommand.doStartLocator(StartLocatorCommand.java:267)
at
org.apache.geode.management.internal.cli.commands.StartLocatorCommand.startLocator(StartLocatorCommand.java:133)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0([email protected]/Native
Method)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke([email protected]/Unknown
Source)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/Unknown
Source)
at java.lang.reflect.Method.invoke([email protected]/Unknown Source)
at
org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
at
org.apache.geode.management.internal.cli.remote.CommandExecutor.callInvokeMethod(CommandExecutor.java:139)
at
org.apache.geode.management.internal.cli.remote.CommandExecutor.invokeCommand(CommandExecutor.java:149)
{code}
was:
We are using gfsh with a script to automatically launch a locator and a server.
During the launch of the locator, the process gfsh just freezes randomly. The
locator process is forked, but gfsh remains frozen.
A jstack on the process shows that the process is stuck on an internal
NativeLibrary.load call. This one happens when gfsh try to attach to the
process : internally the jvm loads the libattach.so library, but the call never
return.
To be complete, Geode is here deployed inside a docker container, using ubi8 as
base image, java is jdk11.0.11 - we have tested with 11.0.13, same issue.
The stack is the following (full jstack dump attached)
"main" #1 prio=5 os_prio=0 cpu=2402.78ms elapsed=1924.45s allocated=236M
defined_classes=10346 tid=0x00007fc904024000 nid=0x16 runnable
[0x00007fc90a84a000]
java.lang.Thread.State: RUNNABLE
at java.lang.ClassLoader$NativeLibrary.load0([email protected]/Native
Method)
at java.lang.ClassLoader$NativeLibrary.load([email protected]/Unknown
Source)
at
java.lang.ClassLoader$NativeLibrary.loadLibrary([email protected]/Unknown
Source)
- locked <0x00000000823efe60> (a java.util.HashSet)
at java.lang.ClassLoader.loadLibrary0([email protected]/Unknown Source)
at java.lang.ClassLoader.loadLibrary([email protected]/Unknown Source)
at java.lang.Runtime.loadLibrary0([email protected]/Unknown Source)
- locked <0x00000000828224b8> (a java.lang.Runtime)
at java.lang.System.loadLibrary([email protected]/Unknown Source)
at sun.tools.attach.VirtualMachineImpl.<clinit>([email protected]/Unknown
Source)
at
sun.tools.attach.AttachProviderImpl.attachVirtualMachine([email protected]/Unknown
Source)
at com.sun.tools.attach.VirtualMachine.attach([email protected]/Unknown
Source)
at
org.apache.geode.internal.process.MBeanProcessController.getJMXServiceURL(MBeanProcessController.java:227)
at
org.apache.geode.internal.process.MBeanProcessController.connect(MBeanProcessController.java:192)
at
org.apache.geode.internal.process.MBeanProcessController.invokeOperationOnTargetMBean(MBeanProcessController.java:159)
at
org.apache.geode.internal.process.MBeanProcessController.status(MBeanProcessController.java:136)
at
org.apache.geode.internal.process.MBeanProcessController.status(MBeanProcessController.java:81)
at
org.apache.geode.internal.process.MBeanOrFileProcessController.status(MBeanOrFileProcessController.java:37)
at
org.apache.geode.distributed.LocatorLauncher.statusWithWorkingDirectory(LocatorLauncher.java:1012)
at
org.apache.geode.distributed.LocatorLauncher.status(LocatorLauncher.java:940)
at
org.apache.geode.distributed.LocatorLauncher$LocatorState.fromDirectory(LocatorLauncher.java:2077)
at
org.apache.geode.management.internal.cli.commands.StartLocatorCommand.doStartLocator(StartLocatorCommand.java:267)
at
org.apache.geode.management.internal.cli.commands.StartLocatorCommand.startLocator(StartLocatorCommand.java:133)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0([email protected]/Native
Method)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke([email protected]/Unknown
Source)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/Unknown
Source)
at java.lang.reflect.Method.invoke([email protected]/Unknown Source)
at
org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
at
org.apache.geode.management.internal.cli.remote.CommandExecutor.callInvokeMethod(CommandExecutor.java:139)
at
org.apache.geode.management.internal.cli.remote.CommandExecutor.invokeCommand(CommandExecutor.java:149)
> Freeze during launch of locator
> -------------------------------
>
> Key: GEODE-10081
> URL: https://issues.apache.org/jira/browse/GEODE-10081
> Project: Geode
> Issue Type: Bug
> Components: gfsh
> Affects Versions: 1.14.2
> Reporter: Julien Gilles
> Priority: Major
> Attachments: jstackout.txt
>
>
> We are using gfsh with a script to automatically launch a locator and a
> server.
> During the launch of the locator, the process gfsh just freezes randomly. The
> locator process is forked, but gfsh remains frozen.
> A jstack on the process shows that the process is stuck on an internal
> NativeLibrary.load call. This one happens when gfsh try to attach to the
> process : internally the jvm loads the libattach.so library, but the call
> never return.
> To be complete, Geode is here deployed inside a docker container, using ubi8
> as base image, java is jdk11.0.11 - we have tested with 11.0.13, same issue.
>
> The stack is the following (full jstack dump attached)
> {code:java}
> "main" #1 prio=5 os_prio=0 cpu=2402.78ms elapsed=1924.45s allocated=236M
> defined_classes=10346 tid=0x00007fc904024000 nid=0x16 runnable
> [0x00007fc90a84a000]
> java.lang.Thread.State: RUNNABLE
> at java.lang.ClassLoader$NativeLibrary.load0([email protected]/Native
> Method)
> at java.lang.ClassLoader$NativeLibrary.load([email protected]/Unknown
> Source)
> at
> java.lang.ClassLoader$NativeLibrary.loadLibrary([email protected]/Unknown
> Source)
> - locked <0x00000000823efe60> (a java.util.HashSet)
> at java.lang.ClassLoader.loadLibrary0([email protected]/Unknown Source)
> at java.lang.ClassLoader.loadLibrary([email protected]/Unknown Source)
> at java.lang.Runtime.loadLibrary0([email protected]/Unknown Source)
> - locked <0x00000000828224b8> (a java.lang.Runtime)
> at java.lang.System.loadLibrary([email protected]/Unknown Source)
> at
> sun.tools.attach.VirtualMachineImpl.<clinit>([email protected]/Unknown
> Source)
> at
> sun.tools.attach.AttachProviderImpl.attachVirtualMachine([email protected]/Unknown
> Source)
> at com.sun.tools.attach.VirtualMachine.attach([email protected]/Unknown
> Source)
> at
> org.apache.geode.internal.process.MBeanProcessController.getJMXServiceURL(MBeanProcessController.java:227)
> at
> org.apache.geode.internal.process.MBeanProcessController.connect(MBeanProcessController.java:192)
> at
> org.apache.geode.internal.process.MBeanProcessController.invokeOperationOnTargetMBean(MBeanProcessController.java:159)
> at
> org.apache.geode.internal.process.MBeanProcessController.status(MBeanProcessController.java:136)
> at
> org.apache.geode.internal.process.MBeanProcessController.status(MBeanProcessController.java:81)
> at
> org.apache.geode.internal.process.MBeanOrFileProcessController.status(MBeanOrFileProcessController.java:37)
> at
> org.apache.geode.distributed.LocatorLauncher.statusWithWorkingDirectory(LocatorLauncher.java:1012)
> at
> org.apache.geode.distributed.LocatorLauncher.status(LocatorLauncher.java:940)
> at
> org.apache.geode.distributed.LocatorLauncher$LocatorState.fromDirectory(LocatorLauncher.java:2077)
> at
> org.apache.geode.management.internal.cli.commands.StartLocatorCommand.doStartLocator(StartLocatorCommand.java:267)
> at
> org.apache.geode.management.internal.cli.commands.StartLocatorCommand.startLocator(StartLocatorCommand.java:133)
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0([email protected]/Native
> Method)
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke([email protected]/Unknown
> Source)
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/Unknown
> Source)
> at java.lang.reflect.Method.invoke([email protected]/Unknown Source)
> at
> org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
> at
> org.apache.geode.management.internal.cli.remote.CommandExecutor.callInvokeMethod(CommandExecutor.java:139)
> at
> org.apache.geode.management.internal.cli.remote.CommandExecutor.invokeCommand(CommandExecutor.java:149)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
