[jira] [Updated] (HUDI-5377) Write call stack information to lock file

2023-02-15 Thread Ethan Guo (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Guo updated HUDI-5377:

Fix Version/s: 0.13.0

> Write call stack information to lock file
> -
>
> Key: HUDI-5377
> URL: https://issues.apache.org/jira/browse/HUDI-5377
> Project: Apache Hudi
>  Issue Type: Improvement
>Reporter: HunterXHunter
>Assignee: HunterXHunter
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.13.0
>
>
> When Occ is enabled, Sometimes an exception is thrown 'Unable  to acquire 
> lock',
> We need to know which step caused the deadlock.
> like :
> {
>   "lockCreateTime" : 1671017890189,
>   "lockStackInfo" : [ "\t java.lang.Thread.getStackTrace (Thread.java:1564) 
> \n", "\t 
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.initLockInfo
>  (FileSystemBasedLockProvider.java:212) \n", "\t 
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.acquireLock
>  (FileSystemBasedLockProvider.java:172) \n", "\t 
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.tryLock 
> (FileSystemBasedLockProvider.java:116) \n", "\t 
> org.apache.hudi.client.transaction.lock.LockManager.lock 
> (LockManager.java:108) \n", "\t 
> org.apache.hudi.client.transaction.TransactionManager.beginTransaction 
> (TransactionManager.java:58) \n", "\t 
> org.apache.hudi.client.BaseHoodieWriteClient.clean 
> (BaseHoodieWriteClient.java:891) \n", "\t 
> org.apache.hudi.client.BaseHoodieWriteClient.clean 
> (BaseHoodieWriteClient.java:858) \n", "\t 
> org.apache.hudi.sink.CleanFunction.lambda$open$0 (CleanFunction.java:67) \n", 
> "\t org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0 
> (NonThrownExecutor.java:130) \n", "\t 
> java.util.concurrent.ThreadPoolExecutor.runWorker 
> (ThreadPoolExecutor.java:1149) \n", "\t 
> java.util.concurrent.ThreadPoolExecutor$Worker.run 
> (ThreadPoolExecutor.java:624) \n", "\t java.lang.Thread.run (Thread.java:750) 
> \n" ],
>   "lockThreadName" : "pool-8-thread-1"
> }



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HUDI-5377) Write call stack information to lock file

2022-12-14 Thread HunterXHunter (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

HunterXHunter updated HUDI-5377:

Description: 
When Occ is enabled, Sometimes an exception is thrown 'Unable  to acquire lock',

We need to know which step caused the deadlock.

like :

{
  "lockCreateTime" : 1671017890189,
  "lockStackInfo" : [ "\t java.lang.Thread.getStackTrace (Thread.java:1564) 
\n", "\t 
org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.initLockInfo
 (FileSystemBasedLockProvider.java:212) \n", "\t 
org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.acquireLock 
(FileSystemBasedLockProvider.java:172) \n", "\t 
org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.tryLock 
(FileSystemBasedLockProvider.java:116) \n", "\t 
org.apache.hudi.client.transaction.lock.LockManager.lock (LockManager.java:108) 
\n", "\t org.apache.hudi.client.transaction.TransactionManager.beginTransaction 
(TransactionManager.java:58) \n", "\t 
org.apache.hudi.client.BaseHoodieWriteClient.clean 
(BaseHoodieWriteClient.java:891) \n", "\t 
org.apache.hudi.client.BaseHoodieWriteClient.clean 
(BaseHoodieWriteClient.java:858) \n", "\t 
org.apache.hudi.sink.CleanFunction.lambda$open$0 (CleanFunction.java:67) \n", 
"\t org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0 
(NonThrownExecutor.java:130) \n", "\t 
java.util.concurrent.ThreadPoolExecutor.runWorker 
(ThreadPoolExecutor.java:1149) \n", "\t 
java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:624) \n", "\t java.lang.Thread.run (Thread.java:750) 
\n" ],
  "lockThreadName" : "pool-8-thread-1"
}

  was:
When Occ is enabled, Sometimes an exception is thrown 'Unable  to acquire lock',

We need to know which step caused the deadlock.

like :

 

LOCK-TIME : 2022-12-13 11:13:15.015
LOCK-STACK-INFO :
     
org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.acquireLock 
(FileSystemBasedLockProvider.java:148)
     
org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.tryLock 
(FileSystemBasedLockProvider.java:100)
     org.apache.hudi.client.transaction.lock.LockManager.lock 
(LockManager.java:102)
     org.apache.hudi.client.transaction.TransactionManager.beginTransaction 
(TransactionManager.java:58)
     org.apache.hudi.client.BaseHoodieWriteClient.scheduleTableService 
(BaseHoodieWriteClient.java:1425)
     org.apache.hudi.client.BaseHoodieWriteClient.scheduleCompactionAtInstant 
(BaseHoodieWriteClient.java:1037)
     org.apache.hudi.util.CompactionUtil.scheduleCompaction 
(CompactionUtil.java:72)
     
org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$notifyCheckpointComplete$2
 (StreamWriteOperatorCoordinator.java:250)
     org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0 
(NonThrownExecutor.java:130)
     java.util.concurrent.ThreadPoolExecutor.runWorker 
(ThreadPoolExecutor.java:1149)
     java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:624)
     java.lang.Thread.run (Thread.java:750)


> Write call stack information to lock file
> -
>
> Key: HUDI-5377
> URL: https://issues.apache.org/jira/browse/HUDI-5377
> Project: Apache Hudi
>  Issue Type: Improvement
>Reporter: HunterXHunter
>Assignee: HunterXHunter
>Priority: Major
>  Labels: pull-request-available
>
> When Occ is enabled, Sometimes an exception is thrown 'Unable  to acquire 
> lock',
> We need to know which step caused the deadlock.
> like :
> {
>   "lockCreateTime" : 1671017890189,
>   "lockStackInfo" : [ "\t java.lang.Thread.getStackTrace (Thread.java:1564) 
> \n", "\t 
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.initLockInfo
>  (FileSystemBasedLockProvider.java:212) \n", "\t 
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.acquireLock
>  (FileSystemBasedLockProvider.java:172) \n", "\t 
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.tryLock 
> (FileSystemBasedLockProvider.java:116) \n", "\t 
> org.apache.hudi.client.transaction.lock.LockManager.lock 
> (LockManager.java:108) \n", "\t 
> org.apache.hudi.client.transaction.TransactionManager.beginTransaction 
> (TransactionManager.java:58) \n", "\t 
> org.apache.hudi.client.BaseHoodieWriteClient.clean 
> (BaseHoodieWriteClient.java:891) \n", "\t 
> org.apache.hudi.client.BaseHoodieWriteClient.clean 
> (BaseHoodieWriteClient.java:858) \n", "\t 
> org.apache.hudi.sink.CleanFunction.lambda$open$0 (CleanFunction.java:67) \n", 
> "\t org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0 
> (NonThrownExecutor.java:130) \n", "\t 
> java.util.concurrent.ThreadPoolExecutor.runWorker 
> (ThreadPoolExecutor.java:1149) \n", "\t 
> java.util.concurrent.ThreadPoolExecutor$Worker.run 
> (ThreadPoolExecutor.java:624) \n", "

[jira] [Updated] (HUDI-5377) Write call stack information to lock file

2022-12-13 Thread HunterXHunter (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

HunterXHunter updated HUDI-5377:

Summary: Write call stack information to lock file  (was: Add call stack 
information to lock file)

> Write call stack information to lock file
> -
>
> Key: HUDI-5377
> URL: https://issues.apache.org/jira/browse/HUDI-5377
> Project: Apache Hudi
>  Issue Type: Improvement
>Reporter: HunterXHunter
>Assignee: HunterXHunter
>Priority: Major
>  Labels: pull-request-available
>
> When Occ is enabled, Sometimes an exception is thrown 'Unable  to acquire 
> lock',
> We need to know which step caused the deadlock.
> like :
>  
> LOCK-TIME : 2022-12-13 11:13:15.015
> LOCK-STACK-INFO :
>      
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.acquireLock
>  (FileSystemBasedLockProvider.java:148)
>      
> org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider.tryLock 
> (FileSystemBasedLockProvider.java:100)
>      org.apache.hudi.client.transaction.lock.LockManager.lock 
> (LockManager.java:102)
>      org.apache.hudi.client.transaction.TransactionManager.beginTransaction 
> (TransactionManager.java:58)
>      org.apache.hudi.client.BaseHoodieWriteClient.scheduleTableService 
> (BaseHoodieWriteClient.java:1425)
>      org.apache.hudi.client.BaseHoodieWriteClient.scheduleCompactionAtInstant 
> (BaseHoodieWriteClient.java:1037)
>      org.apache.hudi.util.CompactionUtil.scheduleCompaction 
> (CompactionUtil.java:72)
>      
> org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$notifyCheckpointComplete$2
>  (StreamWriteOperatorCoordinator.java:250)
>      org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0 
> (NonThrownExecutor.java:130)
>      java.util.concurrent.ThreadPoolExecutor.runWorker 
> (ThreadPoolExecutor.java:1149)
>      java.util.concurrent.ThreadPoolExecutor$Worker.run 
> (ThreadPoolExecutor.java:624)
>      java.lang.Thread.run (Thread.java:750)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)