[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15117671#comment-15117671
]
Purshotam Shah commented on OOZIE-1922:
---
Thanks Rohini for review. Committed to trunk.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922-V2.patch,
> OOZIE-1922-V3.patch, OOZIE-1922.1.patch, OOZIE-1922.2.patch,
> OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15081957#comment-15081957
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1706
.Tests failed: 6
.Tests errors: 0
.The patch failed the following testcases:
. testPurgeXCommandFailed(org.apache.oozie.command.TestPurgeXCommand)
. testPurgeWFWithSubWF1(org.apache.oozie.command.TestPurgeXCommand)
. testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)
.
testMessage_withMixedStatus(org.apache.oozie.command.coord.TestAbandonedCoordChecker)
. testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
.
testPauseBundleAndCoordinator(org.apache.oozie.service.TestPauseTransitService)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/2691/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922-V2.patch,
> OOZIE-1922-V3.patch, OOZIE-1922.1.patch, OOZIE-1922.2.patch,
> OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074231#comment-15074231
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/2690/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922-V2.patch,
> OOZIE-1922-V3.patch, OOZIE-1922.1.patch, OOZIE-1922.2.patch,
> OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074147#comment-15074147
]
Rohini Palaniswamy commented on OOZIE-1922:
---
+1
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922-V2.patch,
> OOZIE-1922-V3.patch, OOZIE-1922.1.patch, OOZIE-1922.2.patch,
> OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074123#comment-15074123
]
Purshotam Shah commented on OOZIE-1922:
---
I fix that. I have also optimized the memory lock condition.
getReadLockCount is enough, it will also give the count of readlock held by
current thread. We don't have to use getReadHoldCount.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922-V2.patch,
> OOZIE-1922.1.patch, OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073385#comment-15073385
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1706
.Tests failed: 7
.Tests errors: 0
.The patch failed the following testcases:
. testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)
. testPurgeXCommandFailed(org.apache.oozie.command.TestPurgeXCommand)
. testPurgeWFWithSubWF1(org.apache.oozie.command.TestPurgeXCommand)
. testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
.
testCoord_throwException(org.apache.oozie.command.coord.TestCoordChangeXCommand)
. testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor)
. testUpdateSLA(org.apache.oozie.sla.TestSLAService)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/2689/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922-V2.patch,
> OOZIE-1922.1.patch, OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073318#comment-15073318
]
Rohini Palaniswamy commented on OOZIE-1922:
---
Thanks. New logic for memory looks solid now.
{code}
+InterProcessReadWriteLock readWriteLock = zkLocks.get(resource);
// This shouldn't happen, but if it does, then we should remove entry from map
+if (readWriteLock == null) {
+return false;
+}
{code}
Above code means that it is not in the hashmap already, so trying to remove
from hashmap does not make sense. if (readWriteLock == null) check should be
before and outside of isLockHeld.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922-V2.patch,
> OOZIE-1922.1.patch, OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073304#comment-15073304
]
Purshotam Shah commented on OOZIE-1922:
---
Yes, you are correct we don't need to use getParticipantNodes().
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073168#comment-15073168
]
Rohini Palaniswamy commented on OOZIE-1922:
---
bq. It's not that simple, mainly for ZKlock. We need to use
getParticipantNodes() in logic
Why do you need getParticipantNodes() in logic. You just need to remove from
local hashmap when no thread in that jvm holds the lock anymore. It should not
matter if another node has a lock.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073149#comment-15073149
]
Purshotam Shah commented on OOZIE-1922:
---
It's not that simple, mainly for ZKlock. We need to use getParticipantNodes()
in logic. This will call ZK every time, why to make unnecessary calls if we are
not going to use it.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073147#comment-15073147
]
Rohini Palaniswamy commented on OOZIE-1922:
---
bq. I don't think we will ever support readlocks. We don't have to convolute
our logic. I will put TODO in code for readlock support.
I would prefer that we rather use the correct logic instead instead of TODO
as it is simple to fix. It is just couple more conditions which do not cost
much. Don't like the idea of breaking it considering that the rest of code for
read locks is good even though not used.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073137#comment-15073137
]
Purshotam Shah commented on OOZIE-1922:
---
{quote}
You have to. If it is read locks, more than one thread can hold the lock.
Currently only write locks are acquired in the code though, but still logic
needs to be correct as there are APIs to acquire read lock.
{quote}
I don't think we will ever support readlocks. We don't have to convolute our
logic. I will put TODO in code for readlock support.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073124#comment-15073124
]
Rohini Palaniswamy commented on OOZIE-1922:
---
bq. Sine we are in release function, which means that lock is acquired by
current thread. So, we don't have to check if lock is held by other thread or
not.
You have to. If it is read locks, more than one thread can hold the lock.
Currently only write locks are acquired in the code though, but still logic
needs to be correct as there are APIs to acquire read lock.
bq. I didn't remove getParticipantNodes because it was already part of the
condition and was not sure if it's break anything.
If you are really concern about performance, we can remove getParticipantNodes
from condition.
Yes. Performance and load on zookeeper is very important.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073111#comment-15073111
]
Purshotam Shah commented on OOZIE-1922:
---
{quote}
This only checks for if there are no waiting threads and if all locks held by
this thread are released before removing the lock. It does not take into
account if any other thread is holding read or write lock. You will also have
to make use of the following APIs in the logic.
{quote}
Sine we are in release function, which means that lock is acquired by current
thread. So, we don't have to check if lock is held by other thread or not.
{quote}
Even though acquire() may not cause a ZK call for a reentrant lock, looking at
the isAcquiredInThisProcess()/getParticipantNodes() APIs used in release()
call, I am concerned that those will be new calls to ZK and might cause
performance degradation if synchronous is removed in OOZIE-2394
{quote}
isAcquiredInThisProcess() doesn't make any ZK call.
{code:title=InterProcessMutex.java}
@Override
public boolean isAcquiredInThisProcess()
{
return (threadData.size() > 0);
}
getParticipantNodes() does. We may not need getParticipantNodes, since lock is
acquired by current host, it won't be to acquire by other hosts.
!readWriteLock.readLock().isAcquiredInThisProcess() &&
!readWriteLock.writeLock().isAcquiredInThisProcess() is enough.
I didn't remove getParticipantNodes because it was already part of the
condition and was not sure if it's break anything.
If you are really concern about performance, we can remove getParticipantNodes
from condition.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068880#comment-15068880
]
Rohini Palaniswamy commented on OOZIE-1922:
---
Couple of additional comments in review board.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Azrael Seoeun
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068851#comment-15068851
]
Rohini Palaniswamy commented on OOZIE-1922:
---
bq. Can you put the revised patch in review board as it is hard to read logic
for the test cases.
Ignore this. See that you have already uploaded the patch to review board.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Azrael Seoeun
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068848#comment-15068848
]
Rohini Palaniswamy commented on OOZIE-1922:
---
See couple of issues with the patch.
1) rwLock.getQueueLength() == 0 && rwLock.getWriteHoldCount() == 0 &&
rwLock.getReadHoldCount() == 0
This only checks for if there are no waiting threads and if all locks held
by this thread are released before removing the lock. It does not take into
account if any other thread is holding read or write lock. You will also have
to make use of the following APIs in the logic.
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantReadWriteLock.html#getReadLockCount()
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantReadWriteLock.html#isWriteLocked()
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantReadWriteLock.html#isWriteLockedByCurrentThread()
2)
{code}
& !readWriteLock.readLock().isAcquiredInThisProcess()
+&&
!readWriteLock.writeLock().isAcquiredInThisProcess()
{code}
- Should be && instead of &.
- isAcquiredInThisProcess stands for if acquired in this JVM. You are trying
to remove it from current node's zkLocks HashMap if the node does not hold lock
anymore. That is good, but lock.getParticipantNodes().size() == 0 condition
will prevent it if another node is hold the lock. So you will have to remove
that condition.
Haven't looked at the test cases yet. Can you put the revised patch in review
board as it is hard to read logic for the test cases.
3) Related to OOZIE-2394 patch:
Even though acquire() may not cause a ZK call for a reentrant lock, looking
at the isAcquiredInThisProcess()/getParticipantNodes() APIs used in release()
call, I am concerned that those will be new calls to ZK and might cause
performance degradation if synchronous is removed in OOZIE-2394. Can you run
one test case enabling debug logging for ZK and see how many calls are made
with and without removal of synchronous. If there is increase by even one ZK
call, I would like to retain the synchronous method. Cleanup of removing very
few lines of simple code is not worth for the cost of a ZK call.
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Azrael Seoeun
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067478#comment-15067478
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1706
.Tests failed: 3
.Tests errors: 0
.The patch failed the following testcases:
. testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)
. testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
. testUpdateSLA(org.apache.oozie.sla.TestSLAService)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/2686/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Azrael Seoeun
> Attachments: OOZIE-1922-V1.patch, OOZIE-1922.1.patch,
> OOZIE-1922.2.patch, OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057594#comment-15057594
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1704
.Tests failed: 8
.Tests errors: 2
.The patch failed the following testcases:
. testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)
. testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
. testUpdateSLA(org.apache.oozie.sla.TestSLAService)
. testMain(org.apache.oozie.action.hadoop.TestHiveMain)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMain)
. testPig_withNullExternalID(org.apache.oozie.action.hadoop.TestPigMain)
. testEmbeddedPigWithinPython(org.apache.oozie.action.hadoop.TestPigMain)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMainWithOldAPI)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/2652/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Azrael Seoeun
> Attachments: OOZIE-1922.1.patch, OOZIE-1922.2.patch,
> OOZIE-1922.3.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591360#comment-14591360
]
Jaydeep Vishwakarma commented on OOZIE-1922:
[~seoeun25], OOZIE-1923 has been closed. Are you planning to update the
patch?
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Azrael
> Attachments: OOZIE-1922.1.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119214#comment-14119214
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/1849/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
> Attachments: OOZIE-1922.1.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110214#comment-14110214
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1510
.Tests failed: 7
.Tests errors: 2
.The patch failed the following testcases:
. testBundleId(org.apache.oozie.servlet.TestBulkMonitorWebServiceAPI)
.
testMemoryUsageAndSpeed(org.apache.oozie.service.TestPartitionDependencyManagerService)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMainWithOldAPI)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMain)
. testEmbeddedPigWithinPython(org.apache.oozie.action.hadoop.TestPigMain)
. testPig_withNullExternalID(org.apache.oozie.action.hadoop.TestPigMain)
. testMain(org.apache.oozie.action.hadoop.TestHiveMain)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/1726/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
> Attachments: OOZIE-1922.1.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107518#comment-14107518
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/1616/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
> Attachments: OOZIE-1922.1.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083850#comment-14083850
]
Hadoop QA commented on OOZIE-1922:
--
Testing JIRA OOZIE-1922
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1507
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
.
testResetSequence_withMultiThread(org.apache.oozie.service.TestZKUUIDService)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/1436/
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
> Attachments: OOZIE-1922.1.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (OOZIE-1922) MemoryLocksService fails if lock is acquired multiple times in same thread and released
[
https://issues.apache.org/jira/browse/OOZIE-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069779#comment-14069779
]
Azrael commented on OOZIE-1922:
---
[~puru] can you review please?
> MemoryLocksService fails if lock is acquired multiple times in same thread
> and released
> ---
>
> Key: OOZIE-1922
> URL: https://issues.apache.org/jira/browse/OOZIE-1922
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
> Attachments: OOZIE-1922.1.patch
>
>
> ReentrantLock can be acquired multiple times in same thread. For multiple
> acquire call, ReentrantLock hold count is incremented by one.
> So if we acquire lock multiple time from same thread, all will be successful
> and hold count is increased for every call.
> But if we release lock, MemoryLocksService ignore the count and deletes the
> lock. Even if it's held by some command.
> Simple step can reproduce it.
> {code}
> service.getWriteLock("test", 5000); //writeHoldCount = 1
> MemoryLockToken lock = (MemoryLockToken)service.getWriteLock("test",
> 5000); //writeHoldCount = 2
> lock.release(); //writeHoldCount = 1
> lock = (MemoryLockToken)service.getWriteLock("test", 5000);
> //writeHoldCount = 1, it should be 2.
> {code}
> {code}
> @Override
> public void release() {
> int val = rwLock.getQueueLength();
> if (val == 0) {
> synchronized (locks) {
> locks.remove(resource);
> }
> }
> lock.unlock();
> }
> }
> {code}
> MemoryLocks should check hold count before removing lock.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
