[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015159#comment-16015159
]
Chia-Ping Tsai commented on HBASE-18019:
bq. consider using "close" instead of clear since we close all the memstore
scanners.
Good point. Will commit patch with this suggestion.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch, HBASE-18019.v1.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014277#comment-16014277
]
Hadoop QA commented on HBASE-18019:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 45s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 88m 36s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 128m 13s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.TestZooKeeper |
| | org.apache.hadoop.hbase.backup.TestIncrementalBackupDeleteTable |
| | org.apache.hadoop.hbase.TestAcidGuarantees |
| | org.apache.hadoop.hbase.backup.TestRestoreBoundaryTests |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868522/HBASE-18019.v1.patch |
| JIRA Issue | HBASE-18019 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 80f8af90cf83 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 62d7323 |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6824/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6824/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6824/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6824/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014129#comment-16014129
]
Ted Yu commented on HBASE-18019:
Suggestion on the subject of this JIRA: consider using "close" instead of clear
since we close all the memstore scanners.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch, HBASE-18019.v1.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014036#comment-16014036
]
Chia-Ping Tsai commented on HBASE-18019:
Will commit it if the QA is fine
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch, HBASE-18019.v1.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013562#comment-16013562
]
Duo Zhang commented on HBASE-18019:
---
{quote}
We will be creating new memstore scanner (after the flush) and that is been
added. Ideally only the snapshot scanner should get cleared but to do that we
will need larger change. We discussed abt this in 17887
{quote}
Got it. No question then. +1.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013555#comment-16013555
]
Chia-Ping Tsai commented on HBASE-18019:
bq. Since CompactingMemstore is now default,
Thanks for the reminder. Will fix it in v1 patch.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013549#comment-16013549
]
Anoop Sam John commented on HBASE-18019:
bq.Is it safe to just remove all the memstore scanners? If the store is being
flushed when we create the scanner, then after flush the snapshot is gone but
we still need to scan the memstore?
We will be creating new memstore scanner (after the flush) and that is been
added. Ideally only the snapshot scanner should get cleared but to do that we
will need larger change. We discussed abt this in 17887
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013545#comment-16013545
]
Anoop Sam John commented on HBASE-18019:
Only the loop order change here? (and a break). So as such there is no issue
in the codebase right. Trying to understand
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013544#comment-16013544
]
ramkrishna.s.vasudevan commented on HBASE-18019:
bq.The new hfiles is added before the snapshot is gone,
True.
Since CompactingMemstore is now default,
bq. // generally we get two memstore scanners from a flush (active + snapshot)
You could have more than one segment. (a nit).
Else LGTM.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013527#comment-16013527
]
Chia-Ping Tsai commented on HBASE-18019:
bq. If the store is being flushed when we create the scanner, then after flush
the snapshot is gone but we still need to scan the memstore?
The new hfiles is added before the snapshot is gone, so we can get data from
the new hfiles even if the snapshot is gone.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013439#comment-16013439
]
Duo Zhang commented on HBASE-18019:
---
Is it safe to just remove all the memstore scanners? If the store is being
flushed when we create the scanner, then after flush the snapshot is gone but
we still need to scan the memstore? Maybe something related to HBASE-17887?
Thanks.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013432#comment-16013432
]
Chia-Ping Tsai commented on HBASE-18019:
All failed tests pass locally.
[~Apache9] Any comment? Thanks
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
> Attachments: HBASE-18019.v0.patch
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013299#comment-16013299
]
Hadoop QA commented on HBASE-18019:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 37s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 20s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
6s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 152m 52s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.quotas.TestQuotaThrottle |
| Timed out junit tests |
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded |
| | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.10.1 Server=1.10.1 Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868403/HBASE-18019.v0.patch |
| JIRA Issue | HBASE-18019 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux d60ed77451ad 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul
24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a8775b1 |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6816/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6816/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6816/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012751#comment-16012751
]
Chia-Ping Tsai commented on HBASE-18019:
Sorry for delay. (I'm working on TestAcid*..) The patch will be done soon.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012484#comment-16012484
]
Duo Zhang commented on HBASE-18019:
---
Any updates here?
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004048#comment-16004048
]
Chia-Ping Tsai commented on HBASE-18019:
I will fix this after resolving HBASE-17887. Otherwise, the TestAcid* will fail
and confuse us.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004027#comment-16004027
]
ramkrishna.s.vasudevan commented on HBASE-18019:
Yeah when this code was return we only had the DefaultMemstore and also
HBASE-17655 was not there. Yes we need to fix it.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18019) Clear redundant memstore scanners
[
https://issues.apache.org/jira/browse/HBASE-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003894#comment-16003894
]
Duo Zhang commented on HBASE-18019:
---
Yeah I also found this problem when implementing HBASE-17917 but I haven't
analyzed it deeply yet.
> Clear redundant memstore scanners
> -
>
> Key: HBASE-18019
> URL: https://issues.apache.org/jira/browse/HBASE-18019
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
>
> The HBASE-17655 remove the MemStoreScanner and it causes that the
> MemStore#getScanner(readpt) returns multi KeyValueScanner which consist of
> active, snapshot and pipeline. But StoreScanner only remove one mem scanner
> when refreshing current scanners.
> {code}
> for (int i = 0; i < currentScanners.size(); i++) {
> if (!currentScanners.get(i).isFileScanner()) {
> currentScanners.remove(i);
> break;
> }
> }
> {code}
> The older scanners kept in the StoreScanner will hinder GC from releasing
> memory and lead to multiple scans on the same data.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
