[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870551#comment-13870551 ] Hudson commented on HBASE-10324: FAILURE: Integrated in HBase-0.98 #77 (See [https://builds.apache.org/job/HBase-0.98/77/]) HBASE-10324 refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model (Tedyu: rev 1557943) * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestDurability.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java * /hbase/branches/0.98/hbase-shell/src/main/ruby/hbase/admin.rb refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: 10324-trunk_v3.patch, HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870653#comment-13870653 ] Feng Honghua commented on HBASE-10324: -- sorry I was out of office today, thanks [~yuzhih...@gmail.com] for making the new trunk patch refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: 10324-trunk_v3.patch, HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870658#comment-13870658 ] Hudson commented on HBASE-10324: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #72 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/72/]) HBASE-10324 refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model (Tedyu: rev 1557943) * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestDurability.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java * /hbase/branches/0.98/hbase-shell/src/main/ruby/hbase/admin.rb refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: 10324-trunk_v3.patch, HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13871456#comment-13871456 ] Hudson commented on HBASE-10324: SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #53 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/53/]) HBASE-10324 refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model (Tedyu: rev 1557939) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestDurability.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java * /hbase/trunk/hbase-shell/src/main/ruby/hbase/admin.rb refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: 10324-trunk_v3.patch, HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869772#comment-13869772 ] Andrew Purtell commented on HBASE-10324: Let's commit this. It needs to go into 0.98 branch also because we incorporated FHH's log changes there too. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869773#comment-13869773 ] Andrew Purtell commented on HBASE-10324: Nit: Remove the '@Deprecated' tags from the renamed methods. The deprecated methods have been effectively removed and replaced with a new API. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870078#comment-13870078 ] Ted Yu commented on HBASE-10324: Some how I got some conflicts applying 10324-trunk_v2.patch : {code} -rw-r--r-- 1 tyu staff 690 Jan 13 14:21 ./hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java.rej -rw-r--r-- 1 tyu staff 2272 Jan 13 14:21 ./hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java.rej -rw-r--r-- 1 tyu staff 1584 Jan 13 14:21 ./hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestDurability.java.rej -rw-r--r-- 1 tyu staff 1271 Jan 13 14:21 ./hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java.rej -rw-r--r-- 1 tyu staff 1913 Jan 13 14:21 ./hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java.rej {code} Honghua: Can you attach patch for trunk, removing @Deprecated for setAsyncLogFlush() ? Thanks refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870410#comment-13870410 ] Ted Yu commented on HBASE-10324: Integrated patch v3 to trunk. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: 10324-trunk_v3.patch, HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870416#comment-13870416 ] Ted Yu commented on HBASE-10324: After verifying that isDeferredLogFlush() and setDeferredLogFlush() are deprecated in 0.96, I integrated patch v3 to 0.98. Thanks for the patch, Honghua. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: 10324-trunk_v3.patch, HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870415#comment-13870415 ] Hadoop QA commented on HBASE-10324: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12622797/10324-trunk_v3.patch against trunk revision . ATTACHMENT ID: 12622797 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 15 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8416//console This message is automatically generated. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Fix For: 0.98.0, 0.99.0 Attachments: 10324-trunk_v3.patch, HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869103#comment-13869103 ] Ted Yu commented on HBASE-10324: {code} - private long lastDeferredTxid; + private long lastNotSyncedTxid; {code} nit: lastNotSyncedTxid - lastUnSyncedTxid {code} @Deprecated - public synchronized boolean isDeferredLogFlush() { + public synchronized boolean isAsyncLogFlush() { {code} isDeferredLogFlush() is deprecated. Should it be removed from trunk ? refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be true 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869110#comment-13869110 ] Hadoop QA commented on HBASE-10324: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12622536/HBASE-10324-trunk_v0.patch against trunk revision . ATTACHMENT ID: 12622536 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 15 new or modified tests. {color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop 1.0 profile. {color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop 1.1 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: + private HRegion createHRegion (byte [] tableName, String callingMethod, HLog log, boolean isAsyncLogFlush) +htd.setAsyncLogFlush(JBoolean.valueOf(arg.delete(DEFERRED_LOG_FLUSH))) if arg[DEFERRED_LOG_FLUSH] +htd.setAsyncLogFlush(JBoolean.valueOf(arg.delete(DEFERRED_LOG_FLUSH))) if arg[DEFERRED_LOG_FLUSH] {color:red}-1 site{color}. The patch appears to cause mvn site goal to fail. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.io.TestHeapSize Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8394//console This message is automatically generated. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be true 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869225#comment-13869225 ] Feng Honghua commented on HBASE-10324: -- Thanks [~yuzhih...@gmail.com]'s review :-) bq.isDeferredLogFlush() is deprecated. Should it be removed from trunk ? HTableDescriptor.isDeferredLogFlush means writes of this kind of table will return without waiting for sync is done. it controls the write semantic the same way before/after the new write thread model(new write thread model doesn't change the semantic of append/sync). but it should be renamed to isAsyncLogFlush() (by implicit async writer-syncer-notifier threads), not deprecated. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be true 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869228#comment-13869228 ] Feng Honghua commented on HBASE-10324: -- the failed unit-test org.apache.hadoop.hbase.io.TestHeapSize can pass in my local test refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be true 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869233#comment-13869233 ] Ted Yu commented on HBASE-10324: One field was removed from HRegion: {code} - private final boolean deferredLogSyncDisabled; {code} You should lower the number of booleans, at line 5291: {code} 5 * Bytes.SIZEOF_BOOLEAN); {code} The test passes when 5 is changed to 4 refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869235#comment-13869235 ] Feng Honghua commented on HBASE-10324: -- bq. You should lower the number of booleans, at line 5291. The test passes when 5 is changed to 4 Yes, this should be changed. Thanks. But it's weird this test can pass locally before this change, I re-ran it several times before this change...:-( refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869245#comment-13869245 ] Hadoop QA commented on HBASE-10324: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12622563/HBASE-10324-trunk_v1.patch against trunk revision . ATTACHMENT ID: 12622563 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 15 new or modified tests. {color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop 1.0 profile. {color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop 1.1 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: + htd.setAsyncLogFlush(JBoolean.valueOf(arg.delete(DEFERRED_LOG_FLUSH))) if arg[DEFERRED_LOG_FLUSH] +htd.setAsyncLogFlush(JBoolean.valueOf(arg.delete(DEFERRED_LOG_FLUSH))) if arg[DEFERRED_LOG_FLUSH] {color:red}-1 site{color}. The patch appears to cause mvn site goal to fail. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.io.TestHeapSize Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8396//console This message is automatically generated. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869266#comment-13869266 ] Hadoop QA commented on HBASE-10324: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12622566/HBASE-10324-trunk_v2.patch against trunk revision . ATTACHMENT ID: 12622566 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 15 new or modified tests. {color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop 1.0 profile. {color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop 1.1 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: + htd.setAsyncLogFlush(JBoolean.valueOf(arg.delete(DEFERRED_LOG_FLUSH))) if arg[DEFERRED_LOG_FLUSH] +htd.setAsyncLogFlush(JBoolean.valueOf(arg.delete(DEFERRED_LOG_FLUSH))) if arg[DEFERRED_LOG_FLUSH] {color:red}-1 site{color}. The patch appears to cause mvn site goal to fail. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8397//console This message is automatically generated. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10324) refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model
[ https://issues.apache.org/jira/browse/HBASE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869269#comment-13869269 ] Ted Yu commented on HBASE-10324: Long line warning was from ruby script. +1 from me. refactor deferred-log-flush/Durability related interface/code/naming to align with changed semantic of the new write thread model - Key: HBASE-10324 URL: https://issues.apache.org/jira/browse/HBASE-10324 Project: HBase Issue Type: Improvement Components: Client, regionserver Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-10324-trunk_v0.patch, HBASE-10324-trunk_v1.patch, HBASE-10324-trunk_v2.patch By the new write thread model introduced by [HBASE-8755|https://issues.apache.org/jira/browse/HBASE-8755], some deferred-log-flush/Durability API/code/names should be change accordingly: 1. no timer-triggered deferred-log-flush since flush is always done by async threads, so configuration 'hbase.regionserver.optionallogflushinterval' is no longer needed 2. the async writer-syncer-notifier threads will always be triggered implicitly, this semantic is that it always holds that 'hbase.regionserver.optionallogflushinterval' 0, so deferredLogSyncDisabled in HRegion.java which affects durability behavior should always be false 3. what HTableDescriptor.isDeferredLogFlush really means is the write can return without waiting for the sync is done, so the interface name should be changed to isAsyncLogFlush/setAsyncLogFlush to reflect their real meaning -- This message was sent by Atlassian JIRA (v6.1.5#6160)