[
https://issues.apache.org/jira/browse/PHOENIX-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
qazwsx updated PHOENIX-7346:
----------------------------
Description:
After HBase is started, some Phoenix functions are unavailable. The logs show
that the region of the Catalog table cannot be brought online. The stack
analysis shows that a deadlock occurs between phoenixDriver and DriverManager
during class initialization.
--------------------------------------
"pool-101-thread-1" #360 prio=5 os_prio=0 cpu=129.59ms elapsed=230121.51s
tid=0x0000ffff78622000 nid=0x39850 in Object.wait() [0x0000ffff2cf6e000]
java.lang.Thread.State: RUNNABLE
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at java.sql.DriverManager$2.run(DriverManager.java:603)
at java.sql.DriverManager$2.run(DriverManager.java:583)
at java.security.AccessController.doPrivileged(Native Method)
at java.sql.DriverManager.loadInitialDrivers(DriverManager.java:583)
at java.sql.DriverManager.<clinit>(DriverManager.java:101)
at org.apache.phoenix.util.QueryUtil.getConnection(QueryUtil.java:422)
at
org.apache.phoenix.util.QueryUtil.getConnectionOnServer(QueryUtil.java:400)
at
org.apache.phoenix.util.QueryUtil.getConnectionOnServer(QueryUtil.java:381)
at
org.apache.phoenix.coprocessor.TaskRegionObserver$SelfHealingTask.run(TaskRegionObserver.java:162)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
-----------------------------------------------------
"RS_OPEN_PRIORITY_REGION-regionserver/ac-2:21302-0" #270 daemon prio=5
os_prio=0 cpu=1600.63ms elapsed=230127.52s tid=0x0000ffff791a3000 nid=0x380dc
in Object.wait() [0x0000ffff3236f000]
java.lang.Thread.State: RUNNABLE
at org.apache.phoenix.jdbc.PhoenixDriver.<clinit>(PhoenixDriver.java:128)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.phoenix.coprocessor.MetaDataRegionObserver.postOpen(MetaDataRegionObserver.java:216)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$2.call(RegionCoprocessorHost.java:524)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$2.call(RegionCoprocessorHost.java:521)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:591)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:664)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postOpen(RegionCoprocessorHost.java:521)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:1114)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:966)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8024)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7983)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7959)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7918)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7874)
at
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:186)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:102)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
was:
After HBase is started, some Phoenix functions are unavailable. The logs show
that the region of the Catalog table cannot be brought online. The stack
analysis shows that a deadlock occurs between phoenixDriver and DriverManager
during class initialization.
!image-2024-06-25-21-48-44-738.png!
!image-2024-06-25-21-48-05-491.png!
> PhoenixDriver and DriverManager deadlocked during class initialization.
> -----------------------------------------------------------------------
>
> Key: PHOENIX-7346
> URL: https://issues.apache.org/jira/browse/PHOENIX-7346
> Project: Phoenix
> Issue Type: Bug
> Reporter: qazwsx
> Priority: Major
> Attachments: image-2024-06-25-21-48-05-491.png
>
>
> After HBase is started, some Phoenix functions are unavailable. The logs show
> that the region of the Catalog table cannot be brought online. The stack
> analysis shows that a deadlock occurs between phoenixDriver and DriverManager
> during class initialization.
>
> --------------------------------------
> "pool-101-thread-1" #360 prio=5 os_prio=0 cpu=129.59ms elapsed=230121.51s
> tid=0x0000ffff78622000 nid=0x39850 in Object.wait() [0x0000ffff2cf6e000]
> java.lang.Thread.State: RUNNABLE
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at java.lang.Class.newInstance(Class.java:442)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at java.sql.DriverManager$2.run(DriverManager.java:603)
> at java.sql.DriverManager$2.run(DriverManager.java:583)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.sql.DriverManager.loadInitialDrivers(DriverManager.java:583)
> at java.sql.DriverManager.<clinit>(DriverManager.java:101)
> at org.apache.phoenix.util.QueryUtil.getConnection(QueryUtil.java:422)
> at
> org.apache.phoenix.util.QueryUtil.getConnectionOnServer(QueryUtil.java:400)
> at
> org.apache.phoenix.util.QueryUtil.getConnectionOnServer(QueryUtil.java:381)
> at
> org.apache.phoenix.coprocessor.TaskRegionObserver$SelfHealingTask.run(TaskRegionObserver.java:162)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
>
> -----------------------------------------------------
> "RS_OPEN_PRIORITY_REGION-regionserver/ac-2:21302-0" #270 daemon prio=5
> os_prio=0 cpu=1600.63ms elapsed=230127.52s tid=0x0000ffff791a3000 nid=0x380dc
> in Object.wait() [0x0000ffff3236f000]
> java.lang.Thread.State: RUNNABLE
> at org.apache.phoenix.jdbc.PhoenixDriver.<clinit>(PhoenixDriver.java:128)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.phoenix.coprocessor.MetaDataRegionObserver.postOpen(MetaDataRegionObserver.java:216)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$2.call(RegionCoprocessorHost.java:524)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$2.call(RegionCoprocessorHost.java:521)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:591)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:664)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postOpen(RegionCoprocessorHost.java:521)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:1114)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:966)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8024)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7983)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7959)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7918)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7874)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:186)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:102)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
