[jira] [Updated] (HBASE-9004) Fix Documentation around Minor compaction and ttl
[ https://issues.apache.org/jira/browse/HBASE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HBASE-9004: Attachment: HBASE-9004-1.patch > Fix Documentation around Minor compaction and ttl > - > > Key: HBASE-9004 > URL: https://issues.apache.org/jira/browse/HBASE-9004 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Elliott Clark >Assignee: Masatake Iwasaki > Attachments: HBASE-9004-0.patch, HBASE-9004-1.patch > > > Minor compactions should be able to delete KeyValues outside of ttl. The > docs currently suggest otherwise. We should bring them in line. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932904#comment-13932904
]
Hudson commented on HBASE-10731:
SUCCESS: Integrated in HBase-TRUNK #5004 (See
[https://builds.apache.org/job/HBase-TRUNK/5004/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576956)
* /hbase/trunk/bin/hbase-cleanup.sh
* /hbase/trunk/bin/master-backup.sh
* /hbase/trunk/bin/regionservers.sh
* /hbase/trunk/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-9004) Fix Documentation around Minor compaction and ttl
[ https://issues.apache.org/jira/browse/HBASE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HBASE-9004: Component/s: documentation > Fix Documentation around Minor compaction and ttl > - > > Key: HBASE-9004 > URL: https://issues.apache.org/jira/browse/HBASE-9004 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Elliott Clark >Assignee: Masatake Iwasaki > Attachments: HBASE-9004-0.patch > > > Minor compactions should be able to delete KeyValues outside of ttl. The > docs currently suggest otherwise. We should bring them in line. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9294) NPE in /rs-status during RS shutdown
[ https://issues.apache.org/jira/browse/HBASE-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932888#comment-13932888 ] Hudson commented on HBASE-9294: --- FAILURE: Integrated in hbase-0.96 #343 (See [https://builds.apache.org/job/hbase-0.96/343/]) HBASE-9294 NPE in /rs-status during RS shutdown (stack: rev 1576951) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSStatusServlet.java > NPE in /rs-status during RS shutdown > > > Key: HBASE-9294 > URL: https://issues.apache.org/jira/browse/HBASE-9294 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.2 >Reporter: Steve Loughran >Assignee: Rekha Joshi >Priority: Minor > Labels: patch > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-9294.1.patch > > > While hitting reload to see when a kill-initiated RS shutdown would make the > Web UI go away, I got a stack trace from an NPE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10733) Shell incr command should allow last parameter (value) to be missing. As documented.
[ https://issues.apache.org/jira/browse/HBASE-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932889#comment-13932889 ] Hudson commented on HBASE-10733: FAILURE: Integrated in hbase-0.96 #343 (See [https://builds.apache.org/job/hbase-0.96/343/]) HBASE-10733 Shell incr command should allow last parameter (value) to be missing. As documented. (stack: rev 1576994) * /hbase/branches/0.96/hbase-shell/src/main/ruby/shell/commands/incr.rb > Shell incr command should allow last parameter (value) to be missing. As > documented. > > > Key: HBASE-10733 > URL: https://issues.apache.org/jira/browse/HBASE-10733 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.2 > > Attachments: HBASE-10733-v0-0.96.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10679) Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
[ https://issues.apache.org/jira/browse/HBASE-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932886#comment-13932886 ] Hudson commented on HBASE-10679: FAILURE: Integrated in hbase-0.96 #343 (See [https://builds.apache.org/job/hbase-0.96/343/]) HBASE-10679 Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region (stack: rev 1576930) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Both clients get wrong scan results if the first scanner expires and the > second scanner is created with the same scannerId on the same region > - > > Key: HBASE-10679 > URL: https://issues.apache.org/jira/browse/HBASE-10679 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Honghua Feng >Assignee: Honghua Feng >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-10679-trunk_v1.patch, HBASE-10679-trunk_v2.patch, > HBASE-10679-trunk_v2.patch, HBASE-10679-trunk_v2.patch > > > The scenario is as below (both Client A and Client B scan against Region R) > # A opens a scanner SA on R, the scannerId is N, it successfully get its > first row "a" > # SA's lease expires and it's removed from scanners > # B opens a scanner SB on R, the scannerId is N too. it successfully get its > first row "m" > # A issues its second scan request with scannerId N, regionserver finds N is > valid scannerId and the region matches too. (since the region is always > online on this regionserver and both two scanners are against it), so it > executes scan request on SB, returns "n" to A -- wrong! (get data from other > scanner, A expects row something like "b" that follows "a") > # B issues its second scan request with scannerId N, regionserver also thinks > it's valid, and executes scan on SB, return "o" to B -- wrong! (should return > "n" but "n" has been scanned out by A just now) > The consequence is both clients get wrong scan results: > # A gets data from scanner created by other client, its own scanner has > expired and removed > # B misses data which should be gotten but has been wrongly scanned out by A > The root cause is scannerId generated by regionserver can't be guaranteed > unique within regionserver's whole lifecycle, *there is only guarantee that > scannerIds of scanners that are currently still valid (not expired) are > unique*, so a same scannerId can present in scanners again after a former > scanner with this scannerId expires and has been removed from scanners. And > if the second scanner is against the same region, the bug arises. > Theoretically, the possibility of above scenario should be very rare(two > consecutive scans on a same region from two different clients get a same > scannerId, and the first expires before the second is created), but it does > can happen, and once it happens, the consequence is severe(all clients > involved get wrong data), and should be extremely hard to diagnose/debug -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932887#comment-13932887
]
Hudson commented on HBASE-10731:
FAILURE: Integrated in hbase-0.96 #343 (See
[https://builds.apache.org/job/hbase-0.96/343/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576958)
* /hbase/branches/0.96/bin/hbase-cleanup.sh
* /hbase/branches/0.96/bin/master-backup.sh
* /hbase/branches/0.96/bin/regionservers.sh
* /hbase/branches/0.96/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932853#comment-13932853
]
Hudson commented on HBASE-10731:
SUCCESS: Integrated in HBase-0.94-JDK7 #83 (See
[https://builds.apache.org/job/HBase-0.94-JDK7/83/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576955)
* /hbase/branches/0.94/bin/master-backup.sh
* /hbase/branches/0.94/bin/regionservers.sh
* /hbase/branches/0.94/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10514) Forward port HBASE-10466, possible data loss when failed flushes
[
https://issues.apache.org/jira/browse/HBASE-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932852#comment-13932852
]
Lars Hofhansl commented on HBASE-10514:
---
lgtm
This is a bit scary:
{code}
+int flushCount = 0;
+while (this.getMemstoreSize().get() > 0) {
+ try {
+if (flushCount++ > 0) {
+ LOG.info("Running extra flush (carrying snapshot?) " + this);
+}
+internalFlushcache(status);
+ } catch (IOException ioe) {
+status.setStatus("Failed flush " + this + ", putting online
again");
+synchronized (writestate) {
+ writestate.writesEnabled = true;
+}
+// Have to throw to upper layers. I can't abort server from here.
+throw ioe;
+ }
+}
{code}
We better be sure we never get the accounting wrong, or we'll see an endless
loop. Will the loop ever have more than two iterations?
If not, we might want to limit the number of iterations to two
{{for (int flushCount = 0; flushCount < 2 && this.getMemstoreSize().get() > 0;
flushCount++)}}
or something like that...?
> Forward port HBASE-10466, possible data loss when failed flushes
>
>
> Key: HBASE-10514
> URL: https://issues.apache.org/jira/browse/HBASE-10514
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: 10514.txt, 10514v2.txt, 10514v3.txt, 10514v3.txt,
> 10514v4.txt
>
>
> Critical data loss issues that we need to ensure are not in branches beyond
> 0.89fb. Assigning myself.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932841#comment-13932841
]
Hudson commented on HBASE-10731:
FAILURE: Integrated in hbase-0.96-hadoop2 #236 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/236/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576958)
* /hbase/branches/0.96/bin/hbase-cleanup.sh
* /hbase/branches/0.96/bin/master-backup.sh
* /hbase/branches/0.96/bin/regionservers.sh
* /hbase/branches/0.96/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10726) Fix java.lang.ArrayIndexOutOfBoundsException in StochasticLoadBalancer$LocalityBasedCandidateGenerator
[
https://issues.apache.org/jira/browse/HBASE-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932845#comment-13932845
]
Hudson commented on HBASE-10726:
FAILURE: Integrated in hbase-0.96-hadoop2 #236 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/236/])
HBASE-10726 Fix java.lang.ArrayIndexOutOfBoundsException in
StochasticLoadBalancer (stack: rev 1576838)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java
> Fix java.lang.ArrayIndexOutOfBoundsException in
> StochasticLoadBalancer$LocalityBasedCandidateGenerator
> --
>
> Key: HBASE-10726
> URL: https://issues.apache.org/jira/browse/HBASE-10726
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.96.2, 0.98.1, 0.99.0, hbase-10070
>
> Attachments: hbase-10726_v1.patch
>
>
> Recent fix in HBASE-10632 to
> StochasticLoadBalancer$LocalityBasedCandidateGenerator revealed another
> issue:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: -1
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$CandidateGenerator.pickRandomRegion(StochasticLoadBalancer.java:405)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:571)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:235)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1465)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1505)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9294) NPE in /rs-status during RS shutdown
[ https://issues.apache.org/jira/browse/HBASE-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932843#comment-13932843 ] Hudson commented on HBASE-9294: --- FAILURE: Integrated in hbase-0.96-hadoop2 #236 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/236/]) HBASE-9294 NPE in /rs-status during RS shutdown (stack: rev 1576951) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSStatusServlet.java > NPE in /rs-status during RS shutdown > > > Key: HBASE-9294 > URL: https://issues.apache.org/jira/browse/HBASE-9294 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.2 >Reporter: Steve Loughran >Assignee: Rekha Joshi >Priority: Minor > Labels: patch > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-9294.1.patch > > > While hitting reload to see when a kill-initiated RS shutdown would make the > Web UI go away, I got a stack trace from an NPE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10733) Shell incr command should allow last parameter (value) to be missing. As documented.
[ https://issues.apache.org/jira/browse/HBASE-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932844#comment-13932844 ] Hudson commented on HBASE-10733: FAILURE: Integrated in hbase-0.96-hadoop2 #236 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/236/]) HBASE-10733 Shell incr command should allow last parameter (value) to be missing. As documented. (stack: rev 1576994) * /hbase/branches/0.96/hbase-shell/src/main/ruby/shell/commands/incr.rb > Shell incr command should allow last parameter (value) to be missing. As > documented. > > > Key: HBASE-10733 > URL: https://issues.apache.org/jira/browse/HBASE-10733 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.2 > > Attachments: HBASE-10733-v0-0.96.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10679) Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
[ https://issues.apache.org/jira/browse/HBASE-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932840#comment-13932840 ] Hudson commented on HBASE-10679: FAILURE: Integrated in hbase-0.96-hadoop2 #236 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/236/]) HBASE-10679 Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region (stack: rev 1576930) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Both clients get wrong scan results if the first scanner expires and the > second scanner is created with the same scannerId on the same region > - > > Key: HBASE-10679 > URL: https://issues.apache.org/jira/browse/HBASE-10679 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Honghua Feng >Assignee: Honghua Feng >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-10679-trunk_v1.patch, HBASE-10679-trunk_v2.patch, > HBASE-10679-trunk_v2.patch, HBASE-10679-trunk_v2.patch > > > The scenario is as below (both Client A and Client B scan against Region R) > # A opens a scanner SA on R, the scannerId is N, it successfully get its > first row "a" > # SA's lease expires and it's removed from scanners > # B opens a scanner SB on R, the scannerId is N too. it successfully get its > first row "m" > # A issues its second scan request with scannerId N, regionserver finds N is > valid scannerId and the region matches too. (since the region is always > online on this regionserver and both two scanners are against it), so it > executes scan request on SB, returns "n" to A -- wrong! (get data from other > scanner, A expects row something like "b" that follows "a") > # B issues its second scan request with scannerId N, regionserver also thinks > it's valid, and executes scan on SB, return "o" to B -- wrong! (should return > "n" but "n" has been scanned out by A just now) > The consequence is both clients get wrong scan results: > # A gets data from scanner created by other client, its own scanner has > expired and removed > # B misses data which should be gotten but has been wrongly scanned out by A > The root cause is scannerId generated by regionserver can't be guaranteed > unique within regionserver's whole lifecycle, *there is only guarantee that > scannerIds of scanners that are currently still valid (not expired) are > unique*, so a same scannerId can present in scanners again after a former > scanner with this scannerId expires and has been removed from scanners. And > if the second scanner is against the same region, the bug arises. > Theoretically, the possibility of above scenario should be very rare(two > consecutive scans on a same region from two different clients get a same > scannerId, and the first expires before the second is created), but it does > can happen, and once it happens, the consequence is severe(all clients > involved get wrong data), and should be extremely hard to diagnose/debug -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10718) TestHLogSplit fails when it sets a KV size to be negative
[
https://issues.apache.org/jira/browse/HBASE-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932842#comment-13932842
]
Hudson commented on HBASE-10718:
FAILURE: Integrated in hbase-0.96-hadoop2 #236 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/236/])
HBASE-10718 TestHLogSplit fails when it sets a KV size to be negative (Esteban
Gutierrez) (apurtell: rev 1576783)
*
/hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
> TestHLogSplit fails when it sets a KV size to be negative
> -
>
> Key: HBASE-10718
> URL: https://issues.apache.org/jira/browse/HBASE-10718
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.98.0, 0.99.0, 0.96.1.1, 0.94.17
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10718.v0.txt, HBASE-10718.v1.txt,
> HBASE-10718.v2.txt, HBASE-10718.v3-0.94.txt, HBASE-10718.v3.txt
>
>
> From [~jdcryans]:
> {code}
> java.lang.NegativeArraySizeException
> at org.apache.hadoop.hbase.KeyValue.readFields(KeyValue.java:2259)
> at org.apache.hadoop.hbase.KeyValue.readFields(KeyValue.java:2266)
> at
> org.apache.hadoop.hbase.codec.KeyValueCodec$KeyValueDecoder.parseCell(KeyValueCodec.java:64)
> at
> org.apache.hadoop.hbase.codec.BaseDecoder.advance(BaseDecoder.java:46)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALEdit.readFields(WALEdit.java:222)
> at
> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2114)
> at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2242)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:245)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:214)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getNextLogLine(HLogSplitter.java:799)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.parseHLog(HLogSplitter.java:727)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:307)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:217)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:180)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit.testMiddleGarbageCorruptionSkipErrorsReadsHalfOfFile(TestHLogSplit.java:363)
> ...
> {code}
> It seems to me that we're reading a negative length which we use to create
> the byte array and since it's not an IOE we don't treat it as a corrupted
> log. I'm surprised that not a single build has failed like this in the past 3
> years.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932833#comment-13932833
]
Hudson commented on HBASE-10731:
SUCCESS: Integrated in HBase-0.94 #1320 (See
[https://builds.apache.org/job/HBase-0.94/1320/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576955)
* /hbase/branches/0.94/bin/master-backup.sh
* /hbase/branches/0.94/bin/regionservers.sh
* /hbase/branches/0.94/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10514) Forward port HBASE-10466, possible data loss when failed flushes
[
https://issues.apache.org/jira/browse/HBASE-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932832#comment-13932832
]
Anoop Sam John commented on HBASE-10514:
Oh.. whatever doubt was there in my mind seems not an issue.. Reading these
lines
{code}
synchronized (writestate) {
// Disable compacting and flushing by background threads for this
// region.
writestate.writesEnabled = false;
LOG.debug("Closing " + this + ": disabling compactions & flushes");
waitForFlushesAndCompactions();
}
// If we were not just flushing, is it worth doing a preflush...one
// that will clear out of the bulk of the memstore before we put up
// the close flag?
if (!abort && worthPreFlushing()) {
{code}
> Forward port HBASE-10466, possible data loss when failed flushes
>
>
> Key: HBASE-10514
> URL: https://issues.apache.org/jira/browse/HBASE-10514
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: 10514.txt, 10514v2.txt, 10514v3.txt, 10514v3.txt,
> 10514v4.txt
>
>
> Critical data loss issues that we need to ensure are not in branches beyond
> 0.89fb. Assigning myself.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10735) [WINDOWS] Set -XX:MaxPermSize for unit tests
[ https://issues.apache.org/jira/browse/HBASE-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10735: -- Attachment: hbase-10735_v1.patch Trivial patch. I'll test it out, and commit. > [WINDOWS] Set -XX:MaxPermSize for unit tests > > > Key: HBASE-10735 > URL: https://issues.apache.org/jira/browse/HBASE-10735 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Trivial > Fix For: 0.96.2, 0.99.0, 0.98.2 > > Attachments: hbase-10735_v1.patch > > > The tests on windows fail with java.lang.OutOfMemoryError: PermGen space. > We have -XX:MaxPermSize=100m for tests on linux. We should just set it the > same in windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10735) [WINDOWS] Set -XX:MaxPermSize for unit tests
[ https://issues.apache.org/jira/browse/HBASE-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10735: -- Priority: Trivial (was: Major) > [WINDOWS] Set -XX:MaxPermSize for unit tests > > > Key: HBASE-10735 > URL: https://issues.apache.org/jira/browse/HBASE-10735 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Trivial > Fix For: 0.96.2, 0.99.0, 0.98.2 > > > The tests on windows fail with java.lang.OutOfMemoryError: PermGen space. > We have -XX:MaxPermSize=100m for tests on linux. We should just set it the > same in windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-10735) [WINDOWS] Set -XX:MaxPermSize for unit tests
Enis Soztutar created HBASE-10735: - Summary: [WINDOWS] Set -XX:MaxPermSize for unit tests Key: HBASE-10735 URL: https://issues.apache.org/jira/browse/HBASE-10735 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.2, 0.99.0, 0.98.2 The tests on windows fail with java.lang.OutOfMemoryError: PermGen space. We have -XX:MaxPermSize=100m for tests on linux. We should just set it the same in windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932826#comment-13932826
]
Hudson commented on HBASE-10731:
FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #49 (See
[https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/49/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576955)
* /hbase/branches/0.94/bin/master-backup.sh
* /hbase/branches/0.94/bin/regionservers.sh
* /hbase/branches/0.94/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10169) Batch coprocessor
[ https://issues.apache.org/jira/browse/HBASE-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932822#comment-13932822 ] Hudson commented on HBASE-10169: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/]) HBASE-10169 Batch coprocessor (Jingcheng Du and Gary Helmling) (apurtell: rev 1576791) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionCoprocessorServiceExec.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpoint.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointNullResponse.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointWithErrors.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestBatchCoprocessorEndpoint.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/protobuf/generated/ColumnAggregationWithErrorsProtos.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/protobuf/generated/ColumnAggregationWithNullResponseProtos.java * /hbase/trunk/hbase-server/src/test/protobuf/ColumnAggregationNullResponseProtocol.proto * /hbase/trunk/hbase-server/src/test/protobuf/ColumnAggregationWithErrorsProtocol.proto > Batch coprocessor > - > > Key: HBASE-10169 > URL: https://issues.apache.org/jira/browse/HBASE-10169 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 0.99.0 >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Fix For: 0.98.1, 0.99.0 > > Attachments: 10169-alternate-5-0.98.patch, > 10169-alternate-5-trunk.patch, Batch Coprocessor Design Document.docx, > HBASE-10169-V2.patch, HBASE-10169-V3.patch, HBASE-10169-V3.patch, > HBASE-10169-V4.patch, HBASE-10169-V5.patch, HBASE-10169-alternate-2.patch, > HBASE-10169-alternate-3.patch, HBASE-10169-alternate-4.patch, > HBASE-10169-alternate.patch, HBASE-10169.patch > > > This is designed to improve the coprocessor invocation in the client side. > Currently the coprocessor invocation is to send a call to each region. If > there’s one region server, and 100 regions are located in this server, each > coprocessor invocation will send 100 calls, each call uses a single thread in > the client side. The threads will run out soon when the coprocessor > invocations are heavy. > In this design, all the calls to the same region server will be grouped into > one in a single coprocessor invocation. This call will be spread into each > region in the server side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10679) Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
[ https://issues.apache.org/jira/browse/HBASE-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932814#comment-13932814 ] Hudson commented on HBASE-10679: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/]) HBASE-10679 Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region (stack: rev 1576928) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Both clients get wrong scan results if the first scanner expires and the > second scanner is created with the same scannerId on the same region > - > > Key: HBASE-10679 > URL: https://issues.apache.org/jira/browse/HBASE-10679 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Honghua Feng >Assignee: Honghua Feng >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-10679-trunk_v1.patch, HBASE-10679-trunk_v2.patch, > HBASE-10679-trunk_v2.patch, HBASE-10679-trunk_v2.patch > > > The scenario is as below (both Client A and Client B scan against Region R) > # A opens a scanner SA on R, the scannerId is N, it successfully get its > first row "a" > # SA's lease expires and it's removed from scanners > # B opens a scanner SB on R, the scannerId is N too. it successfully get its > first row "m" > # A issues its second scan request with scannerId N, regionserver finds N is > valid scannerId and the region matches too. (since the region is always > online on this regionserver and both two scanners are against it), so it > executes scan request on SB, returns "n" to A -- wrong! (get data from other > scanner, A expects row something like "b" that follows "a") > # B issues its second scan request with scannerId N, regionserver also thinks > it's valid, and executes scan on SB, return "o" to B -- wrong! (should return > "n" but "n" has been scanned out by A just now) > The consequence is both clients get wrong scan results: > # A gets data from scanner created by other client, its own scanner has > expired and removed > # B misses data which should be gotten but has been wrongly scanned out by A > The root cause is scannerId generated by regionserver can't be guaranteed > unique within regionserver's whole lifecycle, *there is only guarantee that > scannerIds of scanners that are currently still valid (not expired) are > unique*, so a same scannerId can present in scanners again after a former > scanner with this scannerId expires and has been removed from scanners. And > if the second scanner is against the same region, the bug arises. > Theoretically, the possibility of above scenario should be very rare(two > consecutive scans on a same region from two different clients get a same > scannerId, and the first expires before the second is created), but it does > can happen, and once it happens, the consequence is severe(all clients > involved get wrong data), and should be extremely hard to diagnose/debug -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10718) TestHLogSplit fails when it sets a KV size to be negative
[
https://issues.apache.org/jira/browse/HBASE-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932818#comment-13932818
]
Hudson commented on HBASE-10718:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/])
HBASE-10718 TestHLogSplit fails when it sets a KV size to be negative (Esteban
Gutierrez) (apurtell: rev 1576747)
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
> TestHLogSplit fails when it sets a KV size to be negative
> -
>
> Key: HBASE-10718
> URL: https://issues.apache.org/jira/browse/HBASE-10718
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.98.0, 0.99.0, 0.96.1.1, 0.94.17
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10718.v0.txt, HBASE-10718.v1.txt,
> HBASE-10718.v2.txt, HBASE-10718.v3-0.94.txt, HBASE-10718.v3.txt
>
>
> From [~jdcryans]:
> {code}
> java.lang.NegativeArraySizeException
> at org.apache.hadoop.hbase.KeyValue.readFields(KeyValue.java:2259)
> at org.apache.hadoop.hbase.KeyValue.readFields(KeyValue.java:2266)
> at
> org.apache.hadoop.hbase.codec.KeyValueCodec$KeyValueDecoder.parseCell(KeyValueCodec.java:64)
> at
> org.apache.hadoop.hbase.codec.BaseDecoder.advance(BaseDecoder.java:46)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALEdit.readFields(WALEdit.java:222)
> at
> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2114)
> at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2242)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:245)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:214)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getNextLogLine(HLogSplitter.java:799)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.parseHLog(HLogSplitter.java:727)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:307)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:217)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLog(HLogSplitter.java:180)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit.testMiddleGarbageCorruptionSkipErrorsReadsHalfOfFile(TestHLogSplit.java:363)
> ...
> {code}
> It seems to me that we're reading a negative length which we use to create
> the byte array and since it's not an IOE we don't treat it as a corrupted
> log. I'm surprised that not a single build has failed like this in the past 3
> years.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932815#comment-13932815
]
Hudson commented on HBASE-10731:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576956)
* /hbase/trunk/bin/hbase-cleanup.sh
* /hbase/trunk/bin/master-backup.sh
* /hbase/trunk/bin/regionservers.sh
* /hbase/trunk/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10720) rpcClient: Wrong log level when closing the connection
[
https://issues.apache.org/jira/browse/HBASE-10720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932817#comment-13932817
]
Hudson commented on HBASE-10720:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/])
HBASE-10720 rpcClient: Wrong log level when closing the connection (nkeywal:
rev 1576695)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
> rpcClient: Wrong log level when closing the connection
> --
>
> Key: HBASE-10720
> URL: https://issues.apache.org/jira/browse/HBASE-10720
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.99.0, hbase-10070
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Minor
> Fix For: 0.99.0, hbase-10070
>
> Attachments: 10720.v1.patch
>
>
> {code}
> LOG.warn(getName() + ": marking at should close, reason =" +
> e.getMessage());
> if (LOG.isDebugEnabled()) {
> LOG.debug(getName() + ": marking at should close, reason =" +
> e.getMessage()); <== Once is enough
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10726) Fix java.lang.ArrayIndexOutOfBoundsException in StochasticLoadBalancer$LocalityBasedCandidateGenerator
[
https://issues.apache.org/jira/browse/HBASE-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932820#comment-13932820
]
Hudson commented on HBASE-10726:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/])
HBASE-10726 Fix java.lang.ArrayIndexOutOfBoundsException in
StochasticLoadBalancer (Enis Soztutar) (apurtell: rev 1576795)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java
> Fix java.lang.ArrayIndexOutOfBoundsException in
> StochasticLoadBalancer$LocalityBasedCandidateGenerator
> --
>
> Key: HBASE-10726
> URL: https://issues.apache.org/jira/browse/HBASE-10726
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.96.2, 0.98.1, 0.99.0, hbase-10070
>
> Attachments: hbase-10726_v1.patch
>
>
> Recent fix in HBASE-10632 to
> StochasticLoadBalancer$LocalityBasedCandidateGenerator revealed another
> issue:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: -1
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$CandidateGenerator.pickRandomRegion(StochasticLoadBalancer.java:405)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:571)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:235)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1465)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1505)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9294) NPE in /rs-status during RS shutdown
[ https://issues.apache.org/jira/browse/HBASE-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932819#comment-13932819 ] Hudson commented on HBASE-9294: --- FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/]) HBASE-9294 NPE in /rs-status during RS shutdown (stack: rev 1576953) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSStatusServlet.java > NPE in /rs-status during RS shutdown > > > Key: HBASE-9294 > URL: https://issues.apache.org/jira/browse/HBASE-9294 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.2 >Reporter: Steve Loughran >Assignee: Rekha Joshi >Priority: Minor > Labels: patch > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-9294.1.patch > > > While hitting reload to see when a kill-initiated RS shutdown would make the > Web UI go away, I got a stack trace from an NPE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10732) should be 'drop_namespace', instead of 'delete_namespace'
[
https://issues.apache.org/jira/browse/HBASE-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932816#comment-13932816
]
Hudson commented on HBASE-10732:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/])
HBASE-10732 should be 'drop_namespace', instead of 'delete_namespace' (stack:
rev 1576912)
* /hbase/trunk/src/main/docbkx/book.xml
> should be 'drop_namespace', instead of 'delete_namespace'
> --
>
> Key: HBASE-10732
> URL: https://issues.apache.org/jira/browse/HBASE-10732
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Demai Ni
>Assignee: stack
>Priority: Minor
>
> at http://hbase.apache.org/book/namespace.html,
> the example shows
> {code}
> #delete namespace
> delete_namespace 'my_ns'
> {code}
> should be 'drop_namespace'.
> Not sure how to fix a doc error. Hopefully this is the right way to report
> such issue. Thanks
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10688) Add a draining_node script to manage nodes in draining mode
[ https://issues.apache.org/jira/browse/HBASE-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932821#comment-13932821 ] Hudson commented on HBASE-10688: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #115 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/115/]) HBASE-10688 Add a draining_node script to manage nodes in draining mode (enis: rev 1576856) * /hbase/trunk/bin/draining_servers.rb > Add a draining_node script to manage nodes in draining mode > --- > > Key: HBASE-10688 > URL: https://issues.apache.org/jira/browse/HBASE-10688 > Project: HBase > Issue Type: Sub-task > Components: scripts >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.99.0 > > Attachments: 0001-HBASE-10688-v1.patch > > > As per parent issue, our support for graceful stop and node decommissioning > can be improved a lot. However, until we fix it for good, we can add a > draining_node script to manage the nodes in draining mode in zk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10184) [Online Schema Change]: Add additional tests for online schema change
[ https://issues.apache.org/jira/browse/HBASE-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932807#comment-13932807 ] Andrew Purtell commented on HBASE-10184: >From the JIRA. I will try again later. > [Online Schema Change]: Add additional tests for online schema change > - > > Key: HBASE-10184 > URL: https://issues.apache.org/jira/browse/HBASE-10184 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 0.96.1, 0.99.0, 0.98.2 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: online_schema_change > Attachments: HBASE-10184-trunk.diff > > > There are some gaps in testing for Online Schema Change: > Examples of some tests that should be added: > 1. Splits with online schema change > 2. Merge during online schema change > 3. MR over HBase during online schema change > 4. Bulk Load during online schema change > 5. Online change table owner > 6. Online Replication scope change > 7. Online Bloom Filter change > 8. Snapshots during online schema change (HBASE-10136) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932805#comment-13932805
]
Hudson commented on HBASE-10731:
FAILURE: Integrated in HBase-0.94-security #439 (See
[https://builds.apache.org/job/HBase-0.94-security/439/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576955)
* /hbase/branches/0.94/bin/master-backup.sh
* /hbase/branches/0.94/bin/regionservers.sh
* /hbase/branches/0.94/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10514) Forward port HBASE-10466, possible data loss when failed flushes
[
https://issues.apache.org/jira/browse/HBASE-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932803#comment-13932803
]
Anoop Sam John commented on HBASE-10514:
bq.Only one flush can be going on at a time (you can make a request any time).
This created doubt for me
{code}
-if (!abort && !wasFlushing && worthPreFlushing()) {
+if (!abort && worthPreFlushing()) {
{code}
Checking more closely now Stack.
> Forward port HBASE-10466, possible data loss when failed flushes
>
>
> Key: HBASE-10514
> URL: https://issues.apache.org/jira/browse/HBASE-10514
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: 10514.txt, 10514v2.txt, 10514v3.txt, 10514v3.txt,
> 10514v4.txt
>
>
> Critical data loss issues that we need to ensure are not in branches beyond
> 0.89fb. Assigning myself.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9999) Add support for small reverse scan
[ https://issues.apache.org/jira/browse/HBASE-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932801#comment-13932801 ] Hudson commented on HBASE-: --- SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #210 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/210/]) HBASE- Add support for small reverse scan (Nicolas Liochon) (apurtell: rev 1576976) * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallReversedScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java > Add support for small reverse scan > -- > > Key: HBASE- > URL: https://issues.apache.org/jira/browse/HBASE- > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Nicolas Liochon > Fix For: 0.98.1, 0.99.0 > > Attachments: .v1.patch, .v2.patch, .v3.patch > > > HBASE-4811 adds feature of reverse scan. This JIRA adds the support for small > reverse scan. > This is activated when both 'reversed' and 'small' attributes are true in > Scan Object -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9294) NPE in /rs-status during RS shutdown
[ https://issues.apache.org/jira/browse/HBASE-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932800#comment-13932800 ] Hudson commented on HBASE-9294: --- SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #210 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/210/]) HBASE-9294 NPE in /rs-status during RS shutdown (stack: rev 1576952) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSStatusServlet.java > NPE in /rs-status during RS shutdown > > > Key: HBASE-9294 > URL: https://issues.apache.org/jira/browse/HBASE-9294 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.2 >Reporter: Steve Loughran >Assignee: Rekha Joshi >Priority: Minor > Labels: patch > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-9294.1.patch > > > While hitting reload to see when a kill-initiated RS shutdown would make the > Web UI go away, I got a stack trace from an NPE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10679) Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
[ https://issues.apache.org/jira/browse/HBASE-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932798#comment-13932798 ] Hudson commented on HBASE-10679: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #210 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/210/]) HBASE-10679 Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region (stack: rev 1576929) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Both clients get wrong scan results if the first scanner expires and the > second scanner is created with the same scannerId on the same region > - > > Key: HBASE-10679 > URL: https://issues.apache.org/jira/browse/HBASE-10679 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Honghua Feng >Assignee: Honghua Feng >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-10679-trunk_v1.patch, HBASE-10679-trunk_v2.patch, > HBASE-10679-trunk_v2.patch, HBASE-10679-trunk_v2.patch > > > The scenario is as below (both Client A and Client B scan against Region R) > # A opens a scanner SA on R, the scannerId is N, it successfully get its > first row "a" > # SA's lease expires and it's removed from scanners > # B opens a scanner SB on R, the scannerId is N too. it successfully get its > first row "m" > # A issues its second scan request with scannerId N, regionserver finds N is > valid scannerId and the region matches too. (since the region is always > online on this regionserver and both two scanners are against it), so it > executes scan request on SB, returns "n" to A -- wrong! (get data from other > scanner, A expects row something like "b" that follows "a") > # B issues its second scan request with scannerId N, regionserver also thinks > it's valid, and executes scan on SB, return "o" to B -- wrong! (should return > "n" but "n" has been scanned out by A just now) > The consequence is both clients get wrong scan results: > # A gets data from scanner created by other client, its own scanner has > expired and removed > # B misses data which should be gotten but has been wrongly scanned out by A > The root cause is scannerId generated by regionserver can't be guaranteed > unique within regionserver's whole lifecycle, *there is only guarantee that > scannerIds of scanners that are currently still valid (not expired) are > unique*, so a same scannerId can present in scanners again after a former > scanner with this scannerId expires and has been removed from scanners. And > if the second scanner is against the same region, the bug arises. > Theoretically, the possibility of above scenario should be very rare(two > consecutive scans on a same region from two different clients get a same > scannerId, and the first expires before the second is created), but it does > can happen, and once it happens, the consequence is severe(all clients > involved get wrong data), and should be extremely hard to diagnose/debug -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932799#comment-13932799
]
Hudson commented on HBASE-10731:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #210 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/210/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576957)
* /hbase/branches/0.98/bin/hbase-cleanup.sh
* /hbase/branches/0.98/bin/master-backup.sh
* /hbase/branches/0.98/bin/regionservers.sh
* /hbase/branches/0.98/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10728) get_counter value is never used.
[
https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932787#comment-13932787
]
Hadoop QA commented on HBASE-10728:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634310/HBASE-10728-v1-trunk.patch
against trunk revision .
ATTACHMENT ID: 12634310
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8963//console
This message is automatically generated.
> get_counter value is never used.
>
>
> Key: HBASE-10728
> URL: https://issues.apache.org/jira/browse/HBASE-10728
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.2, 0.98.1, 0.99.0
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
> Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch,
> HBASE-10728-v0-trunk.patch, HBASE-10728-v1-0.96.patch,
> HBASE-10728-v1-0.98.patch, HBASE-10728-v1-trunk.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10648) Pluggable Memstore
[ https://issues.apache.org/jira/browse/HBASE-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932786#comment-13932786 ] Anoop Sam John commented on HBASE-10648: May be better I will wait for ur commit (HBASE-10514) > Pluggable Memstore > -- > > Key: HBASE-10648 > URL: https://issues.apache.org/jira/browse/HBASE-10648 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Attachments: HBASE-10648.patch, HBASE-10648_V2.patch, > HBASE-10648_V3.patch, HBASE-10648_V4.patch > > > Make Memstore into an interface implementation. Also make it pluggable by > configuring the FQCN of the impl. > This will allow us to have different impl and optimizations in the Memstore > DataStructure and the upper layers untouched. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10648) Pluggable Memstore
[ https://issues.apache.org/jira/browse/HBASE-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932784#comment-13932784 ] Anoop Sam John commented on HBASE-10648: bq.You looking for more compliments? Don't be greedy now... Hey no boss.. ;) The SnapshotInfo prt comment is reworked and related changes. So thought you will be interested in reviewing those changes. Thanks for the review Stack. Will commit later today IST. Before that if HBASE-10514 can go in, I will make the necessary change in this patch also. > Pluggable Memstore > -- > > Key: HBASE-10648 > URL: https://issues.apache.org/jira/browse/HBASE-10648 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Attachments: HBASE-10648.patch, HBASE-10648_V2.patch, > HBASE-10648_V3.patch, HBASE-10648_V4.patch > > > Make Memstore into an interface implementation. Also make it pluggable by > configuring the FQCN of the impl. > This will allow us to have different impl and optimizations in the Memstore > DataStructure and the upper layers untouched. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10476) HBase Master log grows very fast after stopped hadoop (due to connection exception)
[
https://issues.apache.org/jira/browse/HBASE-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932778#comment-13932778
]
haosdent commented on HBASE-10476:
--
{quote}
You ok w/ our committing w/o test given above?
{quote}
I am ok, thanks. I also try to add a test to mock this scenario, but I failed.
[~nidmhbase] Thank you very much for your patch. I would try this in our
cluster. :-)
> HBase Master log grows very fast after stopped hadoop (due to connection
> exception)
> ---
>
> Key: HBASE-10476
> URL: https://issues.apache.org/jira/browse/HBASE-10476
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 0.96.2, 0.98.1, 0.99.0
>
> Attachments: 10476v2.txt, HBASE-10476-trunk-v0.patch,
> HBASE-10476-trunk-v0.patch, HBASE-10476-trunk-v1.patch
>
>
> hbase 96.0(probably the same issue on 94.x) on single node cluster. At some
> point, we stopped Hadoop, but keep hbase running. As expected, hbase began to
> throw connection errors.
> P.S., later testing shows that this problem doesn't limit to single node
> cluster.
> For the first hour, the regionserver log grows by ~10MB, and master log
> doesn't grow much, which is ok.
> {code:title=log size after one hour}
> -rw-rw-r-- 1 biadmin biadmin 497959 Feb 5 10:36
> hbase-biadmin-master-hdtest014.svl.ibm.com.log
> ...
> -rw-rw-r-- 1 biadmin biadmin 8865371 Feb 5 10:37
> hbase-biadmin-regionserver-hdtest014.svl.ibm.com.log
> {code}
> However, within 4 hours, the Master log grows to 13GB. And it only stops due
> to out of disk space.
> {code:title=log size after 4 hour}
> -rw-rw-r-- 1 biadmin biadmin 3521880064 Feb 5 14:10
> hbase-biadmin-master-hdtest014.svl.ibm.com.log
> -rw-rw-r-- 1 biadmin biadmin 10737418582 Feb 5 11:25
> hbase-biadmin-master-hdtest014.svl.ibm.com.log.1
> ...
> -rw-rw-r-- 1 biadmin biadmin11222365 Feb 5 10:49
> hbase-biadmin-regionserver-hdtest014.svl.ibm.com.log
> {code}
> The exception/error message filled out Master log is
> {code:title=Error message filling up Master log}
> 2014-02-05 11:37:48,688 INFO
> org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler: Splitting
> hbase:meta logs for hdtest014.svl.ibm.com,60020,1391622549030
> 2014-02-05 11:37:48,689 ERROR org.apache.hadoop.hbase.executor.EventHandler:
> Caught throwable while processing event M_META_SERVER_SHUTDOWN
> java.io.IOException: failed log splitting for
> hdtest014.svl.ibm.com,60020,1391622549030, will retry
> at
> org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler.process(MetaServerShutdownHandler.java:70)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:906)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:929)
> at java.lang.Thread.run(Thread.java:738)
> Caused by: java.net.ConnectException: Call From
> hdtest014.svl.ibm.com/9.30.194.23 to hdtest014.svl.ibm.com:9000 failed on
> connection exception: java.net.ConnectException: Connection refused; For more
> details see: http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:39)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:527)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1351)
> at org.apache.hadoop.ipc.Client.call(Client.java:1300)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Method.invoke(Method.java:611)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:651)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Method.in
[jira] [Commented] (HBASE-9800) Impl CoprocessorRowcounter to run in command-line
[
https://issues.apache.org/jira/browse/HBASE-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932777#comment-13932777
]
Ted Yu commented on HBASE-9800:
---
Year is not needed in license header.
{code}
+ public static class LocalFileSink implements Sink{
{code}
Where is LocalFileSink used ?
{code}
+ Put put = new Put(Bytes.toBytes(tableName+"_"+date));
{code}
Why is date needed in row key ?
Can you attach patch for trunk ?
> Impl CoprocessorRowcounter to run in command-line
> -
>
> Key: HBASE-9800
> URL: https://issues.apache.org/jira/browse/HBASE-9800
> Project: HBase
> Issue Type: New Feature
> Components: Coprocessors
>Affects Versions: 0.94.3
>Reporter: chendihao
>Assignee: chendihao
>Priority: Minor
> Attachments: HBASE-9800-0.94-v1.patch
>
>
> We want to count the rows of table daily but the default rowcounter using
> mapreduce is inefficient. Impl rowcounter using coprocessor makes it better.
> Furthermore, It must provide the mechanism to choose the way to output result.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9294) NPE in /rs-status during RS shutdown
[ https://issues.apache.org/jira/browse/HBASE-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932774#comment-13932774 ] Hudson commented on HBASE-9294: --- SUCCESS: Integrated in HBase-0.98 #224 (See [https://builds.apache.org/job/HBase-0.98/224/]) HBASE-9294 NPE in /rs-status during RS shutdown (stack: rev 1576952) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSStatusServlet.java > NPE in /rs-status during RS shutdown > > > Key: HBASE-9294 > URL: https://issues.apache.org/jira/browse/HBASE-9294 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.2 >Reporter: Steve Loughran >Assignee: Rekha Joshi >Priority: Minor > Labels: patch > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-9294.1.patch > > > While hitting reload to see when a kill-initiated RS shutdown would make the > Web UI go away, I got a stack trace from an NPE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9999) Add support for small reverse scan
[ https://issues.apache.org/jira/browse/HBASE-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932775#comment-13932775 ] Hudson commented on HBASE-: --- SUCCESS: Integrated in HBase-0.98 #224 (See [https://builds.apache.org/job/HBase-0.98/224/]) HBASE- Add support for small reverse scan (Nicolas Liochon) (apurtell: rev 1576976) * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallReversedScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientSmallScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java * /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java > Add support for small reverse scan > -- > > Key: HBASE- > URL: https://issues.apache.org/jira/browse/HBASE- > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Nicolas Liochon > Fix For: 0.98.1, 0.99.0 > > Attachments: .v1.patch, .v2.patch, .v3.patch > > > HBASE-4811 adds feature of reverse scan. This JIRA adds the support for small > reverse scan. > This is activated when both 'reversed' and 'small' attributes are true in > Scan Object -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10679) Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
[ https://issues.apache.org/jira/browse/HBASE-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932772#comment-13932772 ] Hudson commented on HBASE-10679: SUCCESS: Integrated in HBase-0.98 #224 (See [https://builds.apache.org/job/HBase-0.98/224/]) HBASE-10679 Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region (stack: rev 1576929) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Both clients get wrong scan results if the first scanner expires and the > second scanner is created with the same scannerId on the same region > - > > Key: HBASE-10679 > URL: https://issues.apache.org/jira/browse/HBASE-10679 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Honghua Feng >Assignee: Honghua Feng >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-10679-trunk_v1.patch, HBASE-10679-trunk_v2.patch, > HBASE-10679-trunk_v2.patch, HBASE-10679-trunk_v2.patch > > > The scenario is as below (both Client A and Client B scan against Region R) > # A opens a scanner SA on R, the scannerId is N, it successfully get its > first row "a" > # SA's lease expires and it's removed from scanners > # B opens a scanner SB on R, the scannerId is N too. it successfully get its > first row "m" > # A issues its second scan request with scannerId N, regionserver finds N is > valid scannerId and the region matches too. (since the region is always > online on this regionserver and both two scanners are against it), so it > executes scan request on SB, returns "n" to A -- wrong! (get data from other > scanner, A expects row something like "b" that follows "a") > # B issues its second scan request with scannerId N, regionserver also thinks > it's valid, and executes scan on SB, return "o" to B -- wrong! (should return > "n" but "n" has been scanned out by A just now) > The consequence is both clients get wrong scan results: > # A gets data from scanner created by other client, its own scanner has > expired and removed > # B misses data which should be gotten but has been wrongly scanned out by A > The root cause is scannerId generated by regionserver can't be guaranteed > unique within regionserver's whole lifecycle, *there is only guarantee that > scannerIds of scanners that are currently still valid (not expired) are > unique*, so a same scannerId can present in scanners again after a former > scanner with this scannerId expires and has been removed from scanners. And > if the second scanner is against the same region, the bug arises. > Theoretically, the possibility of above scenario should be very rare(two > consecutive scans on a same region from two different clients get a same > scannerId, and the first expires before the second is created), but it does > can happen, and once it happens, the consequence is severe(all clients > involved get wrong data), and should be extremely hard to diagnose/debug -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932773#comment-13932773
]
Hudson commented on HBASE-10731:
SUCCESS: Integrated in HBase-0.98 #224 (See
[https://builds.apache.org/job/HBase-0.98/224/])
HBASE-10731 Fix environment variables typos in scripts (Albert Chu) (jmhsieh:
rev 1576957)
* /hbase/branches/0.98/bin/hbase-cleanup.sh
* /hbase/branches/0.98/bin/master-backup.sh
* /hbase/branches/0.98/bin/regionservers.sh
* /hbase/branches/0.98/bin/rolling-restart.sh
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-9979) TestUsersOperationsWithSecureHadoop should be medium test
[
https://issues.apache.org/jira/browse/HBASE-9979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9979:
--
Resolution: Later
Status: Resolved (was: Patch Available)
> TestUsersOperationsWithSecureHadoop should be medium test
> -
>
> Key: HBASE-9979
> URL: https://issues.apache.org/jira/browse/HBASE-9979
> Project: HBase
> Issue Type: Test
>Affects Versions: 0.96.1.1
>Reporter: Ted Yu
>Priority: Minor
> Labels: test
> Attachments: HBASE-9979.1.patch
>
>
> From https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/838/console :
> {code}
> Running org.apache.hadoop.hbase.security.TestUsersOperationsWithSecureHadoop
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 152.96 sec
> {code}
> TestUsersOperationsWithSecureHadoop should be marked as medium test
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10616) Integration test for multi-get calls
[
https://issues.apache.org/jira/browse/HBASE-10616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj Das updated HBASE-10616:
Attachment: 10616-3.txt
This patch should address your comments. BTW on the keysForThisReader not
getting enough keys, I don't think it's an issue. Assume you have 40 threads
and some keys, the first reader will get {0,40,80}, and the second reader
will get {1,41,...} and so on. So all keys are covered. Am i missing something?
> Integration test for multi-get calls
>
>
> Key: HBASE-10616
> URL: https://issues.apache.org/jira/browse/HBASE-10616
> Project: HBase
> Issue Type: Sub-task
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Attachments: 10616-1.txt, 10616-2.2.txt, 10616-3.txt
>
>
> HBASE-10572 adds a test that does 'get' from client. We should add a similar
> test for batch gets.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-4920) We need a mascot, a totem
[ https://issues.apache.org/jira/browse/HBASE-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932744#comment-13932744 ] Roman Shaposhnik commented on HBASE-4920: - [~saint@gmail.com] yup either google docs or survey monkey. And good call on combinations. > We need a mascot, a totem > - > > Key: HBASE-4920 > URL: https://issues.apache.org/jira/browse/HBASE-4920 > Project: HBase > Issue Type: Task >Reporter: stack > Attachments: HBase Orca Logo.jpg, Orca_479990801.jpg, Screen shot > 2011-11-30 at 4.06.17 PM.png, apache hbase orca logo_Proof 3.pdf, apache > logo_Proof 8.pdf, krake.zip, more_orcas.png, more_orcas2.png, photo (2).JPG, > plus_orca.png > > > We need a totem for our t-shirt that is yet to be printed. O'Reilly owns the > Clyesdale. We need something else. > We could have a fluffy little duck that quacks 'hbase!' when you squeeze it > and we could order boxes of them from some off-shore sweatshop that > subcontracts to a contractor who employs child labor only. > Or we could have an Orca (Big!, Fast!, Killer!, and in a poem that Marcy from > Salesforce showed me, that was a bit too spiritual for me to be seen quoting > here, it had the Orca as the 'Guardian of the Cosmic Memory': i.e. in > translation, bigdata). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10184) [Online Schema Change]: Add additional tests for online schema change
[ https://issues.apache.org/jira/browse/HBASE-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932733#comment-13932733 ] Aleksandr Shulman commented on HBASE-10184: --- Thanks for following up on this. Was the patch that you applied the one from the JIRA, or was it the latest from the reviewboard review: That one is here: https://reviews.apache.org/r/16457/diff/raw/ > [Online Schema Change]: Add additional tests for online schema change > - > > Key: HBASE-10184 > URL: https://issues.apache.org/jira/browse/HBASE-10184 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 0.96.1, 0.99.0, 0.98.2 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: online_schema_change > Attachments: HBASE-10184-trunk.diff > > > There are some gaps in testing for Online Schema Change: > Examples of some tests that should be added: > 1. Splits with online schema change > 2. Merge during online schema change > 3. MR over HBase during online schema change > 4. Bulk Load during online schema change > 5. Online change table owner > 6. Online Replication scope change > 7. Online Bloom Filter change > 8. Snapshots during online schema change (HBASE-10136) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10184) [Online Schema Change]: Add additional tests for online schema change
[ https://issues.apache.org/jira/browse/HBASE-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-10184: --- Affects Version/s: (was: 0.98.1) 0.98.2 TestRowCounter with a schema change fails reliably, moving out of 0.98.1 > [Online Schema Change]: Add additional tests for online schema change > - > > Key: HBASE-10184 > URL: https://issues.apache.org/jira/browse/HBASE-10184 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 0.96.1, 0.99.0, 0.98.2 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: online_schema_change > Attachments: HBASE-10184-trunk.diff > > > There are some gaps in testing for Online Schema Change: > Examples of some tests that should be added: > 1. Splits with online schema change > 2. Merge during online schema change > 3. MR over HBase during online schema change > 4. Bulk Load during online schema change > 5. Online change table owner > 6. Online Replication scope change > 7. Online Bloom Filter change > 8. Snapshots during online schema change (HBASE-10136) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10476) HBase Master log grows very fast after stopped hadoop (due to connection exception)
[
https://issues.apache.org/jira/browse/HBASE-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932718#comment-13932718
]
stack commented on HBASE-10476:
---
No worries [~nidmhbase] I can fix on commit.
[~haosd...@gmail.com] You ok w/ our committing w/o test given above?
> HBase Master log grows very fast after stopped hadoop (due to connection
> exception)
> ---
>
> Key: HBASE-10476
> URL: https://issues.apache.org/jira/browse/HBASE-10476
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 0.96.2, 0.98.1, 0.99.0
>
> Attachments: 10476v2.txt, HBASE-10476-trunk-v0.patch,
> HBASE-10476-trunk-v0.patch, HBASE-10476-trunk-v1.patch
>
>
> hbase 96.0(probably the same issue on 94.x) on single node cluster. At some
> point, we stopped Hadoop, but keep hbase running. As expected, hbase began to
> throw connection errors.
> P.S., later testing shows that this problem doesn't limit to single node
> cluster.
> For the first hour, the regionserver log grows by ~10MB, and master log
> doesn't grow much, which is ok.
> {code:title=log size after one hour}
> -rw-rw-r-- 1 biadmin biadmin 497959 Feb 5 10:36
> hbase-biadmin-master-hdtest014.svl.ibm.com.log
> ...
> -rw-rw-r-- 1 biadmin biadmin 8865371 Feb 5 10:37
> hbase-biadmin-regionserver-hdtest014.svl.ibm.com.log
> {code}
> However, within 4 hours, the Master log grows to 13GB. And it only stops due
> to out of disk space.
> {code:title=log size after 4 hour}
> -rw-rw-r-- 1 biadmin biadmin 3521880064 Feb 5 14:10
> hbase-biadmin-master-hdtest014.svl.ibm.com.log
> -rw-rw-r-- 1 biadmin biadmin 10737418582 Feb 5 11:25
> hbase-biadmin-master-hdtest014.svl.ibm.com.log.1
> ...
> -rw-rw-r-- 1 biadmin biadmin11222365 Feb 5 10:49
> hbase-biadmin-regionserver-hdtest014.svl.ibm.com.log
> {code}
> The exception/error message filled out Master log is
> {code:title=Error message filling up Master log}
> 2014-02-05 11:37:48,688 INFO
> org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler: Splitting
> hbase:meta logs for hdtest014.svl.ibm.com,60020,1391622549030
> 2014-02-05 11:37:48,689 ERROR org.apache.hadoop.hbase.executor.EventHandler:
> Caught throwable while processing event M_META_SERVER_SHUTDOWN
> java.io.IOException: failed log splitting for
> hdtest014.svl.ibm.com,60020,1391622549030, will retry
> at
> org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler.process(MetaServerShutdownHandler.java:70)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:906)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:929)
> at java.lang.Thread.run(Thread.java:738)
> Caused by: java.net.ConnectException: Call From
> hdtest014.svl.ibm.com/9.30.194.23 to hdtest014.svl.ibm.com:9000 failed on
> connection exception: java.net.ConnectException: Connection refused; For more
> details see: http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:39)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:527)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1351)
> at org.apache.hadoop.ipc.Client.call(Client.java:1300)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Method.invoke(Method.java:611)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:651)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Method.invoke(Method.java:611)
> at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:266)
> at com.sun.pro
[jira] [Commented] (HBASE-9294) NPE in /rs-status during RS shutdown
[ https://issues.apache.org/jira/browse/HBASE-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932711#comment-13932711 ] Hudson commented on HBASE-9294: --- SUCCESS: Integrated in HBase-TRUNK #5003 (See [https://builds.apache.org/job/HBase-TRUNK/5003/]) HBASE-9294 NPE in /rs-status during RS shutdown (stack: rev 1576953) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSStatusServlet.java > NPE in /rs-status during RS shutdown > > > Key: HBASE-9294 > URL: https://issues.apache.org/jira/browse/HBASE-9294 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.2 >Reporter: Steve Loughran >Assignee: Rekha Joshi >Priority: Minor > Labels: patch > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-9294.1.patch > > > While hitting reload to see when a kill-initiated RS shutdown would make the > Web UI go away, I got a stack trace from an NPE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8410) Basic quota support for namespaces
[
https://issues.apache.org/jira/browse/HBASE-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932708#comment-13932708
]
stack commented on HBASE-8410:
--
Can we also post the patch on rb? Thanks.
> Basic quota support for namespaces
> --
>
> Key: HBASE-8410
> URL: https://issues.apache.org/jira/browse/HBASE-8410
> Project: HBase
> Issue Type: Sub-task
>Reporter: Francis Liu
>Assignee: Vandana Ayyalasomayajula
> Attachments: HBASE-8410_trunk_10.patch, HBASE-8410_trunk_10.patch,
> HBASE-8410_trunk_11.patch, HBASE-8410_trunk_12.patch,
> HBASE-8410_trunk_13.patch, HBASE-8410_trunk_14.patch,
> HBASE-8410_trunk_2.patch, HBASE-8410_trunk_3.patch, HBASE-8410_trunk_4.patch,
> HBASE-8410_trunk_4.patch, HBASE-8410_trunk_5.patch, HBASE-8410_trunk_6.patch,
> HBASE-8410_trunk_7.patch, HBASE-8410_trunk_8.patch, HBASE-8410_trunk_9.patch,
> HBASE_8410.patch, HBASE_8410_1_trunk.patch
>
>
> This task involves creating an observer which provides basic quota support to
> namespaces in terms of (1) number of tables and (2) number of regions. The
> quota support can be enabled by setting:
>
> hbase.coprocessor.region.classes
> org.apache.hadoop.hbase.namespace.NamespaceController
>
>
> hbase.coprocessor.master.classes
> org.apache.hadoop.hbase.namespace.NamespaceController
>
> in the hbase-site.xml.
> To add quotas to namespace, while creating namespace properties need to be
> added.
> Examples:
> 1. namespace_create 'ns1', {'hbase.namespace.quota.maxregion'=>'10'}
> 2. 1. namespace_create 'ns2', {'hbase.namespace.quota.maxtables'=>'2'},
> {'hbase.namespace.quota.maxregion'=>'5'}
> The quotas can be modified/added to namespace at any point of time.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10688) Add a draining_node script to manage nodes in draining mode
[ https://issues.apache.org/jira/browse/HBASE-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932712#comment-13932712 ] Hudson commented on HBASE-10688: SUCCESS: Integrated in HBase-TRUNK #5003 (See [https://builds.apache.org/job/HBase-TRUNK/5003/]) HBASE-10688 Add a draining_node script to manage nodes in draining mode (enis: rev 1576856) * /hbase/trunk/bin/draining_servers.rb > Add a draining_node script to manage nodes in draining mode > --- > > Key: HBASE-10688 > URL: https://issues.apache.org/jira/browse/HBASE-10688 > Project: HBase > Issue Type: Sub-task > Components: scripts >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.99.0 > > Attachments: 0001-HBASE-10688-v1.patch > > > As per parent issue, our support for graceful stop and node decommissioning > can be improved a lot. However, until we fix it for good, we can add a > draining_node script to manage the nodes in draining mode in zk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10732) should be 'drop_namespace', instead of 'delete_namespace'
[
https://issues.apache.org/jira/browse/HBASE-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932710#comment-13932710
]
Hudson commented on HBASE-10732:
SUCCESS: Integrated in HBase-TRUNK #5003 (See
[https://builds.apache.org/job/HBase-TRUNK/5003/])
HBASE-10732 should be 'drop_namespace', instead of 'delete_namespace' (stack:
rev 1576912)
* /hbase/trunk/src/main/docbkx/book.xml
> should be 'drop_namespace', instead of 'delete_namespace'
> --
>
> Key: HBASE-10732
> URL: https://issues.apache.org/jira/browse/HBASE-10732
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Demai Ni
>Assignee: stack
>Priority: Minor
>
> at http://hbase.apache.org/book/namespace.html,
> the example shows
> {code}
> #delete namespace
> delete_namespace 'my_ns'
> {code}
> should be 'drop_namespace'.
> Not sure how to fix a doc error. Hopefully this is the right way to report
> such issue. Thanks
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10679) Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
[ https://issues.apache.org/jira/browse/HBASE-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932709#comment-13932709 ] Hudson commented on HBASE-10679: SUCCESS: Integrated in HBase-TRUNK #5003 (See [https://builds.apache.org/job/HBase-TRUNK/5003/]) HBASE-10679 Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region (stack: rev 1576928) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Both clients get wrong scan results if the first scanner expires and the > second scanner is created with the same scannerId on the same region > - > > Key: HBASE-10679 > URL: https://issues.apache.org/jira/browse/HBASE-10679 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua >Assignee: Feng Honghua >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-10679-trunk_v1.patch, HBASE-10679-trunk_v2.patch, > HBASE-10679-trunk_v2.patch, HBASE-10679-trunk_v2.patch > > > The scenario is as below (both Client A and Client B scan against Region R) > # A opens a scanner SA on R, the scannerId is N, it successfully get its > first row "a" > # SA's lease expires and it's removed from scanners > # B opens a scanner SB on R, the scannerId is N too. it successfully get its > first row "m" > # A issues its second scan request with scannerId N, regionserver finds N is > valid scannerId and the region matches too. (since the region is always > online on this regionserver and both two scanners are against it), so it > executes scan request on SB, returns "n" to A -- wrong! (get data from other > scanner, A expects row something like "b" that follows "a") > # B issues its second scan request with scannerId N, regionserver also thinks > it's valid, and executes scan on SB, return "o" to B -- wrong! (should return > "n" but "n" has been scanned out by A just now) > The consequence is both clients get wrong scan results: > # A gets data from scanner created by other client, its own scanner has > expired and removed > # B misses data which should be gotten but has been wrongly scanned out by A > The root cause is scannerId generated by regionserver can't be guaranteed > unique within regionserver's whole lifecycle, *there is only guarantee that > scannerIds of scanners that are currently still valid (not expired) are > unique*, so a same scannerId can present in scanners again after a former > scanner with this scannerId expires and has been removed from scanners. And > if the second scanner is against the same region, the bug arises. > Theoretically, the possibility of above scenario should be very rare(two > consecutive scans on a same region from two different clients get a same > scannerId, and the first expires before the second is created), but it does > can happen, and once it happens, the consequence is severe(all clients > involved get wrong data), and should be extremely hard to diagnose/debug -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8410) Basic quota support for namespaces
[
https://issues.apache.org/jira/browse/HBASE-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932706#comment-13932706
]
stack commented on HBASE-8410:
--
Formatting is odd in ProtobufUtil
Formatting is odd in general in this patch. Please see rest of hbase code (two
spaces for a tab, etc.)
Any chance of a high-level writeup on how this patch works before start in on a
review?
Does the patch apply still?
Thanks.
> Basic quota support for namespaces
> --
>
> Key: HBASE-8410
> URL: https://issues.apache.org/jira/browse/HBASE-8410
> Project: HBase
> Issue Type: Sub-task
>Reporter: Francis Liu
>Assignee: Vandana Ayyalasomayajula
> Attachments: HBASE-8410_trunk_10.patch, HBASE-8410_trunk_10.patch,
> HBASE-8410_trunk_11.patch, HBASE-8410_trunk_12.patch,
> HBASE-8410_trunk_13.patch, HBASE-8410_trunk_14.patch,
> HBASE-8410_trunk_2.patch, HBASE-8410_trunk_3.patch, HBASE-8410_trunk_4.patch,
> HBASE-8410_trunk_4.patch, HBASE-8410_trunk_5.patch, HBASE-8410_trunk_6.patch,
> HBASE-8410_trunk_7.patch, HBASE-8410_trunk_8.patch, HBASE-8410_trunk_9.patch,
> HBASE_8410.patch, HBASE_8410_1_trunk.patch
>
>
> This task involves creating an observer which provides basic quota support to
> namespaces in terms of (1) number of tables and (2) number of regions. The
> quota support can be enabled by setting:
>
> hbase.coprocessor.region.classes
> org.apache.hadoop.hbase.namespace.NamespaceController
>
>
> hbase.coprocessor.master.classes
> org.apache.hadoop.hbase.namespace.NamespaceController
>
> in the hbase-site.xml.
> To add quotas to namespace, while creating namespace properties need to be
> added.
> Examples:
> 1. namespace_create 'ns1', {'hbase.namespace.quota.maxregion'=>'10'}
> 2. 1. namespace_create 'ns2', {'hbase.namespace.quota.maxtables'=>'2'},
> {'hbase.namespace.quota.maxregion'=>'5'}
> The quotas can be modified/added to namespace at any point of time.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10531) Revisit how the key byte[] is passed to HFileScanner.seekTo and reseekTo
[
https://issues.apache.org/jira/browse/HBASE-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932698#comment-13932698
]
stack commented on HBASE-10531:
---
So not copying is faster using YCSB?
You need to add good tests for the additions to CellComparator.
bq. +public int compareFlatKey(Cell left, Cell right) {
What is a 'flat' key? Why do we have a Cell compare in KV? Should it be in
CellUtil or CellComparator? Or this is how KV is now? It takes Cells? It
seems so.
Here too. Why in KV are we talking Cells?
-public int compareTimestamps(final KeyValue left, final KeyValue right) {
+public int compareTimestamps(final Cell left, final Cell right) {
... and here:
-public int compareRows(final KeyValue left, final KeyValue right) {
+public int compareRows(final Cell left, final Cell right) {
That said, nice to see our use of Cell in KV compares.
But I'd say KV methods should take KVs, not Cells? If it takes Cells, could be
comparing a non-KV and it could actually work? Is this what we want? Maybe
this is just uglyness left over from how KV has been used/abused up to this?
But I'm thinking these compares would be Cell compares out in a CellUtil or
CellCompare class?
You have long lines in here Mr. [~ram_krish]
You should mark these with @VisibleForTesting
+ // Only for test case purpose
It is good that the byte compares added are not public.
Shouldn't this be unsupportedoperationexception in your new key only class?
+public byte[] getValueArray() {
+ return HConstants.EMPTY_BYTE_ARRAY;
+}
Same for value offset and length.
Tags too?
What is difference between KeyAloneKeyValue and a copy of the original Key but
with no value? I am wary introducing new type? Or, a new type that just threw
UnsupportedOperationException when you tried to get a value...
Why we have to create new key when we pass to a comparator? Is it because we
need to parse the bytes so can pass in an Object? That makes sense. I see now
why it is needed. Suggest rename it as KeyOnlyKeyValue rather than KeyAlongKV.
Also, the inner class needs a bit of a class comment on why it is needed:
i.e. you have a byte array but comparator wants a Cell of some type rather
than parse whole KV byte buffer This is used in the case where we have key
only bytes; i.e. the block index?
Is passing 0 correct in the below?
+ return comparator.compareFlatKey(key,
+ new KeyValue.KeyAloneKeyValue(current.keyBuffer, 0,
current.keyLength));
Should it be an offset? We do this '0' in a few places.
Yeah, what is a 'flat' key? Is it key only?
So, this is the replacement: seekToKeyInBlock ? Purge the old stuff
We have tests for the above?
Should this be a CellComparator rather than a KVComparator:
+
+public int compareKey(KVComparator comparator, Cell key);
(Sorry if you answered this already).
Needs doc: + public static int binarySearch(byte[][] arr, Cell leftCell,
RawComparator comparator) {
The array of byte arrays has Cells in it or it seems KVs?Will it always be
arrays of byte arrays? Or will the binary search be in a block? Or is the
'arr' here a block?
Formatting:
-forceBeforeOnExactMatch);
-}else{
+ forceBeforeOnExactMatch);
+} else {
return seekToOrBeforeUsingPositionAtOrAfter(keyOnlyBytes, offset, length,
-forceBeforeOnExactMatch);
+ forceBeforeOnExactMatch);
We creating a KV where we did not before here?
-return this.delegate.seekBefore(key, offset, length);
+return seekBefore(new KeyValue.KeyAloneKeyValue(key, offset, length));
Or is it that I am just misreading? (It is being created elsewhere anyways)
Why we add these byte-based methods?
+ public int reseekTo(byte[] key) throws IOException {
+ public int reseekTo(byte[] key, int offset, int length)
We should let this out?
+ public org.apache.hadoop.hbase.io.hfile.HFile.Reader getReader() {
Any chance of micro benchmarks on difference between this patch and what was
there before?
Why we adding a method that passes key as bytes?
+public BlockWithScanInfo loadDataBlockWithScanInfo(final byte[] key, int
keyOffset,
+int keyLength, HFileBlock currentBlock, boolean cacheBlocks,
+boolean pread, boolean isCompaction)
+throws IOException {
The ByteBuffers here come from where?
+static int locateNonRootIndexEntry(ByteBuffer nonRootBlock, Cell key,
KVComparator comparator) {
+ int entryIndex = binarySearchNonRootIndex(key, nonRootBlock, comparator);
Yeah, why you add these byte array versions?
+public boolean seekBefore(byte[] key, int offset, int length) throws
IOException {
+ HFileBlock seekToBlock =
reader.getDataBlockIndexReader().seekToDataBlock(key, offset,
+ length, block, cacheBlocks, pread, isCompaction);
Is there duplicated code betw
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Release Note: (was: One more try ;) I simply removed the "value" parameter. Now it's aligned to the documentation.) > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch, HBASE-10728-v1-0.96.patch, > HBASE-10728-v1-0.98.patch, HBASE-10728-v1-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932690#comment-13932690 ] Jean-Marc Spaggiari commented on HBASE-10728: - One more try ;) I simply removed the "value" parameter. Now it's aligned to the documentation. Tested it in 0.96 only, but it's the exact same file for the 3 so should work on all. > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch, HBASE-10728-v1-0.96.patch, > HBASE-10728-v1-0.98.patch, HBASE-10728-v1-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Release Note: One more try ;) I simply removed the "value" parameter. Now it's aligned to the documentation. Status: Patch Available (was: Open) > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch, HBASE-10728-v1-0.96.patch, > HBASE-10728-v1-0.98.patch, HBASE-10728-v1-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9004) Fix Documentation around Minor compaction and ttl
[
https://issues.apache.org/jira/browse/HBASE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932687#comment-13932687
]
Hadoop QA commented on HBASE-9004:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634265/HBASE-9004-0.patch
against trunk revision .
ATTACHMENT ID: 12634265
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+lead position of the key (e.g., "" to
""). Running those key ranges through Bytes.split
+ See http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HColumnDescriptor.html";>HColumnDescriptor
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8962//console
This message is automatically generated.
> Fix Documentation around Minor compaction and ttl
> -
>
> Key: HBASE-9004
> URL: https://issues.apache.org/jira/browse/HBASE-9004
> Project: HBase
> Issue Type: Task
>Reporter: Elliott Clark
>Assignee: Masatake Iwasaki
> Attachments: HBASE-9004-0.patch
>
>
> Minor compactions should be able to delete KeyValues outside of ttl. The
> docs currently suggest otherwise. We should bring them in line.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Attachment: HBASE-10728-v1-trunk.patch > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch, HBASE-10728-v1-0.96.patch, > HBASE-10728-v1-0.98.patch, HBASE-10728-v1-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Attachment: HBASE-10728-v1-0.98.patch > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch, HBASE-10728-v1-0.96.patch, > HBASE-10728-v1-0.98.patch, HBASE-10728-v1-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Attachment: HBASE-10728-v1-0.96.patch > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch, HBASE-10728-v1-0.96.patch, > HBASE-10728-v1-0.98.patch, HBASE-10728-v1-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10476) HBase Master log grows very fast after stopped hadoop (due to connection exception)
[
https://issues.apache.org/jira/browse/HBASE-10476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932684#comment-13932684
]
Demai Ni commented on HBASE-10476:
--
I looked at the javadoc warning. Besides the known ones from Bytes.java, a
couple more was triggerred by
{code}
[WARNING] Javadoc Warnings
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionCoprocessorServiceExec.java:43:
warning - Tag @link: can't find batchCoprocessorService(MethodDescriptor,
Message, byte[], byte[],
[WARNING] Message, Batch.Callback) in org.apache.hadoop.hbase.client.HTable
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionCoprocessorServiceExec.java:43:
warning - Tag @link: can't find batchCoprocessorService(MethodDescriptor,
Message, byte[], byte[], Message) in org.apache.hadoop.hbase.client.HTable
{code}
that is coming from another jira hbase-10169
> HBase Master log grows very fast after stopped hadoop (due to connection
> exception)
> ---
>
> Key: HBASE-10476
> URL: https://issues.apache.org/jira/browse/HBASE-10476
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Demai Ni
>Assignee: Demai Ni
> Fix For: 0.96.2, 0.98.1, 0.99.0
>
> Attachments: 10476v2.txt, HBASE-10476-trunk-v0.patch,
> HBASE-10476-trunk-v0.patch, HBASE-10476-trunk-v1.patch
>
>
> hbase 96.0(probably the same issue on 94.x) on single node cluster. At some
> point, we stopped Hadoop, but keep hbase running. As expected, hbase began to
> throw connection errors.
> P.S., later testing shows that this problem doesn't limit to single node
> cluster.
> For the first hour, the regionserver log grows by ~10MB, and master log
> doesn't grow much, which is ok.
> {code:title=log size after one hour}
> -rw-rw-r-- 1 biadmin biadmin 497959 Feb 5 10:36
> hbase-biadmin-master-hdtest014.svl.ibm.com.log
> ...
> -rw-rw-r-- 1 biadmin biadmin 8865371 Feb 5 10:37
> hbase-biadmin-regionserver-hdtest014.svl.ibm.com.log
> {code}
> However, within 4 hours, the Master log grows to 13GB. And it only stops due
> to out of disk space.
> {code:title=log size after 4 hour}
> -rw-rw-r-- 1 biadmin biadmin 3521880064 Feb 5 14:10
> hbase-biadmin-master-hdtest014.svl.ibm.com.log
> -rw-rw-r-- 1 biadmin biadmin 10737418582 Feb 5 11:25
> hbase-biadmin-master-hdtest014.svl.ibm.com.log.1
> ...
> -rw-rw-r-- 1 biadmin biadmin11222365 Feb 5 10:49
> hbase-biadmin-regionserver-hdtest014.svl.ibm.com.log
> {code}
> The exception/error message filled out Master log is
> {code:title=Error message filling up Master log}
> 2014-02-05 11:37:48,688 INFO
> org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler: Splitting
> hbase:meta logs for hdtest014.svl.ibm.com,60020,1391622549030
> 2014-02-05 11:37:48,689 ERROR org.apache.hadoop.hbase.executor.EventHandler:
> Caught throwable while processing event M_META_SERVER_SHUTDOWN
> java.io.IOException: failed log splitting for
> hdtest014.svl.ibm.com,60020,1391622549030, will retry
> at
> org.apache.hadoop.hbase.master.handler.MetaServerShutdownHandler.process(MetaServerShutdownHandler.java:70)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:906)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:929)
> at java.lang.Thread.run(Thread.java:738)
> Caused by: java.net.ConnectException: Call From
> hdtest014.svl.ibm.com/9.30.194.23 to hdtest014.svl.ibm.com:9000 failed on
> connection exception: java.net.ConnectException: Connection refused; For more
> details see: http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:39)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:527)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1351)
> at org.apache.hadoop.ipc.Client.call(Client.java:1300)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Meth
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Status: Patch Available (was: Open) Very small fixes attached to this patch including 0.96, 0.98 and trunk. > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Status: Open (was: Patch Available) > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932679#comment-13932679 ] Jean-Marc Spaggiari commented on HBASE-10728: - Oups, need a small update... Cancelled. > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Attachment: HBASE-10728-v0-0.98.patch > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Attachment: HBASE-10728-v0-trunk.patch > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch, HBASE-10728-v0-0.98.patch, > HBASE-10728-v0-trunk.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10514) Forward port HBASE-10466, possible data loss when failed flushes
[ https://issues.apache.org/jira/browse/HBASE-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932677#comment-13932677 ] Andrew Purtell commented on HBASE-10514: bq. Just need review. Jetlagged, back tomorrow. > Forward port HBASE-10466, possible data loss when failed flushes > > > Key: HBASE-10514 > URL: https://issues.apache.org/jira/browse/HBASE-10514 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18 > > Attachments: 10514.txt, 10514v2.txt, 10514v3.txt, 10514v3.txt, > 10514v4.txt > > > Critical data loss issues that we need to ensure are not in branches beyond > 0.89fb. Assigning myself. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10679) Both clients get wrong scan results if the first scanner expires and the second scanner is created with the same scannerId on the same region
[ https://issues.apache.org/jira/browse/HBASE-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932674#comment-13932674 ] Andrew Purtell commented on HBASE-10679: +1, thanks! > Both clients get wrong scan results if the first scanner expires and the > second scanner is created with the same scannerId on the same region > - > > Key: HBASE-10679 > URL: https://issues.apache.org/jira/browse/HBASE-10679 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Feng Honghua >Assignee: Feng Honghua >Priority: Critical > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: HBASE-10679-trunk_v1.patch, HBASE-10679-trunk_v2.patch, > HBASE-10679-trunk_v2.patch, HBASE-10679-trunk_v2.patch > > > The scenario is as below (both Client A and Client B scan against Region R) > # A opens a scanner SA on R, the scannerId is N, it successfully get its > first row "a" > # SA's lease expires and it's removed from scanners > # B opens a scanner SB on R, the scannerId is N too. it successfully get its > first row "m" > # A issues its second scan request with scannerId N, regionserver finds N is > valid scannerId and the region matches too. (since the region is always > online on this regionserver and both two scanners are against it), so it > executes scan request on SB, returns "n" to A -- wrong! (get data from other > scanner, A expects row something like "b" that follows "a") > # B issues its second scan request with scannerId N, regionserver also thinks > it's valid, and executes scan on SB, return "o" to B -- wrong! (should return > "n" but "n" has been scanned out by A just now) > The consequence is both clients get wrong scan results: > # A gets data from scanner created by other client, its own scanner has > expired and removed > # B misses data which should be gotten but has been wrongly scanned out by A > The root cause is scannerId generated by regionserver can't be guaranteed > unique within regionserver's whole lifecycle, *there is only guarantee that > scannerIds of scanners that are currently still valid (not expired) are > unique*, so a same scannerId can present in scanners again after a former > scanner with this scannerId expires and has been removed from scanners. And > if the second scanner is against the same region, the bug arises. > Theoretically, the possibility of above scenario should be very rare(two > consecutive scans on a same region from two different clients get a same > scannerId, and the first expires before the second is created), but it does > can happen, and once it happens, the consequence is severe(all clients > involved get wrong data), and should be extremely hard to diagnose/debug -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Attachment: HBASE-10728-v0-0.96.patch > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-10728-v0-0.96.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10728) get_counter value is never used.
[ https://issues.apache.org/jira/browse/HBASE-10728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10728: Affects Version/s: 0.99.0 0.98.1 0.96.2 > get_counter value is never used. > > > Key: HBASE-10728 > URL: https://issues.apache.org/jira/browse/HBASE-10728 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.2, 0.98.1, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10734) Fix RegionStates.getRegionAssignments to not add duplicate regions
[ https://issues.apache.org/jira/browse/HBASE-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HBASE-10734. - Resolution: Fixed Committed. > Fix RegionStates.getRegionAssignments to not add duplicate regions > -- > > Key: HBASE-10734 > URL: https://issues.apache.org/jira/browse/HBASE-10734 > Project: HBase > Issue Type: Sub-task >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10734-1.txt > > > RegionStates.getRegionAssignments(Collection) erroneously adds the > default replica twice to the value list in the returned map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10733) Shell incr command should allow last parameter (value) to be missing. As documented.
[ https://issues.apache.org/jira/browse/HBASE-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932636#comment-13932636 ] Jean-Marc Spaggiari commented on HBASE-10733: - Yep, only required in 0.96. all the 3 other branches are correct. Thanks for the commit. > Shell incr command should allow last parameter (value) to be missing. As > documented. > > > Key: HBASE-10733 > URL: https://issues.apache.org/jira/browse/HBASE-10733 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.2 > > Attachments: HBASE-10733-v0-0.96.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10733) Shell incr command should allow last parameter (value) to be missing. As documented.
[ https://issues.apache.org/jira/browse/HBASE-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-10733. --- Resolution: Fixed Fix Version/s: 0.96.2 Hadoop Flags: Reviewed Committed to 0.96. Thats it [~jmspaggi]? > Shell incr command should allow last parameter (value) to be missing. As > documented. > > > Key: HBASE-10733 > URL: https://issues.apache.org/jira/browse/HBASE-10733 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.2 > > Attachments: HBASE-10733-v0-0.96.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10733) Shell incr command should allow last parameter (value) to be missing. As documented.
[ https://issues.apache.org/jira/browse/HBASE-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10733: Attachment: HBASE-10733-v0-0.96.patch Simple patch attached. [~stack], up to you to push it into 0.96. I tested it again 0.96.1.1-hadoop2. > Shell incr command should allow last parameter (value) to be missing. As > documented. > > > Key: HBASE-10733 > URL: https://issues.apache.org/jira/browse/HBASE-10733 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Attachments: HBASE-10733-v0-0.96.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10184) [Online Schema Change]: Add additional tests for online schema change
[ https://issues.apache.org/jira/browse/HBASE-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932594#comment-13932594 ] Andrew Purtell commented on HBASE-10184: Existing patch applies to trunk and 0.98 with minor fuzz only. Looping tests for a bit. > [Online Schema Change]: Add additional tests for online schema change > - > > Key: HBASE-10184 > URL: https://issues.apache.org/jira/browse/HBASE-10184 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 0.96.1, 0.98.1, 0.99.0 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: online_schema_change > Attachments: HBASE-10184-trunk.diff > > > There are some gaps in testing for Online Schema Change: > Examples of some tests that should be added: > 1. Splits with online schema change > 2. Merge during online schema change > 3. MR over HBase during online schema change > 4. Bulk Load during online schema change > 5. Online change table owner > 6. Online Replication scope change > 7. Online Bloom Filter change > 8. Snapshots during online schema change (HBASE-10136) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10734) Fix RegionStates.getRegionAssignments to not add duplicate regions
[ https://issues.apache.org/jira/browse/HBASE-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932596#comment-13932596 ] Enis Soztutar commented on HBASE-10734: --- +1. Can you please run TestStochasticBalancer and TestBaseLoadBalancer before commit. > Fix RegionStates.getRegionAssignments to not add duplicate regions > -- > > Key: HBASE-10734 > URL: https://issues.apache.org/jira/browse/HBASE-10734 > Project: HBase > Issue Type: Sub-task >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10734-1.txt > > > RegionStates.getRegionAssignments(Collection) erroneously adds the > default replica twice to the value list in the returned map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10380) Add bytesBinary and filter options to CopyTable
[ https://issues.apache.org/jira/browse/HBASE-10380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-10380: --- Status: Open (was: Patch Available) > Add bytesBinary and filter options to CopyTable > --- > > Key: HBASE-10380 > URL: https://issues.apache.org/jira/browse/HBASE-10380 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Ishan Chhabra >Assignee: Ishan Chhabra >Priority: Minor > Attachments: HBASE_10380_0.94-v1.patch > > > Add options in CopyTable to: > 1. Specify the start and stop row in "bytesBinary" format > 2. Use filters -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10734) Fix RegionStates.getRegionAssignments to not add duplicate regions
[ https://issues.apache.org/jira/browse/HBASE-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10734: -- Fix Version/s: hbase-10070 > Fix RegionStates.getRegionAssignments to not add duplicate regions > -- > > Key: HBASE-10734 > URL: https://issues.apache.org/jira/browse/HBASE-10734 > Project: HBase > Issue Type: Sub-task >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10734-1.txt > > > RegionStates.getRegionAssignments(Collection) erroneously adds the > default replica twice to the value list in the returned map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-9999) Add support for small reverse scan
[ https://issues.apache.org/jira/browse/HBASE-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-: -- Fix Version/s: 0.98.1 Committed to 0.98. > Add support for small reverse scan > -- > > Key: HBASE- > URL: https://issues.apache.org/jira/browse/HBASE- > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Nicolas Liochon > Fix For: 0.98.1, 0.99.0 > > Attachments: .v1.patch, .v2.patch, .v3.patch > > > HBASE-4811 adds feature of reverse scan. This JIRA adds the support for small > reverse scan. > This is activated when both 'reversed' and 'small' attributes are true in > Scan Object -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10620) LoadBalancer.needsBalance() should check for co-located region replicas as well
[ https://issues.apache.org/jira/browse/HBASE-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932574#comment-13932574 ] Enis Soztutar commented on HBASE-10620: --- Thanks Devaraj. I committed the addendum patch. > LoadBalancer.needsBalance() should check for co-located region replicas as > well > --- > > Key: HBASE-10620 > URL: https://issues.apache.org/jira/browse/HBASE-10620 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10620-1.txt, 10620-2.txt, 10620-3-addendum.patch, > 10620-3.txt > > > This is a left over TODO from reviews of HBASE-10351. > LB.needsBalance() does some basic checking before running the LB.balance() > method. We need to check whether there are co-located regions in this method > so that the balancer can increase availability. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10726) Fix java.lang.ArrayIndexOutOfBoundsException in StochasticLoadBalancer$LocalityBasedCandidateGenerator
[
https://issues.apache.org/jira/browse/HBASE-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932567#comment-13932567
]
Hudson commented on HBASE-10726:
FAILURE: Integrated in hbase-0.96 #342 (See
[https://builds.apache.org/job/hbase-0.96/342/])
HBASE-10726 Fix java.lang.ArrayIndexOutOfBoundsException in
StochasticLoadBalancer (stack: rev 1576838)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java
> Fix java.lang.ArrayIndexOutOfBoundsException in
> StochasticLoadBalancer$LocalityBasedCandidateGenerator
> --
>
> Key: HBASE-10726
> URL: https://issues.apache.org/jira/browse/HBASE-10726
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.96.2, 0.98.1, 0.99.0, hbase-10070
>
> Attachments: hbase-10726_v1.patch
>
>
> Recent fix in HBASE-10632 to
> StochasticLoadBalancer$LocalityBasedCandidateGenerator revealed another
> issue:
> {code}
> java.lang.ArrayIndexOutOfBoundsException: -1
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$CandidateGenerator.pickRandomRegion(StochasticLoadBalancer.java:405)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:571)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:235)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1465)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1505)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932563#comment-13932563 ] stack commented on HBASE-9117: -- Sounds reasonable. > Remove HTablePool and all HConnection pooling related APIs > -- > > Key: HBASE-9117 > URL: https://issues.apache.org/jira/browse/HBASE-9117 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Nick Dimiduk >Priority: Critical > Fix For: 0.99.0 > > Attachments: HBASE-9117.00.patch, HBASE-9117.01.patch, > HBASE-9117.02.patch, HBASE-9117.03.patch, HBASE-9117.04.patch, > HBASE-9117.05.patch, HBASE-9117.06.patch > > > The recommended way is now: > # Create an HConnection: HConnectionManager.createConnection(...) > # Create a light HTable: HConnection.getTable(...) > # table.close() > # connection.close() > All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10729) Enable table doesn't balance out replicas evenly if the replicas were unassigned earlier
[ https://issues.apache.org/jira/browse/HBASE-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HBASE-10729. - Resolution: Fixed > Enable table doesn't balance out replicas evenly if the replicas were > unassigned earlier > > > Key: HBASE-10729 > URL: https://issues.apache.org/jira/browse/HBASE-10729 > Project: HBase > Issue Type: Sub-task >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10729-1.txt > > > Enable table doesn't assign out replicas keeping availability in mind, if the > replicas were unassigned before the table was disabled. For example, when a > snapshot is restored and then the table is enabled, the replicas are unevenly > assigned. The reason for this is that the the enable table invokes > randomAssign that assigns one region at a time. Since the master doesn't have > any information about the unassigned replicas, the calls to randomAssign > can't do any availability checks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932555#comment-13932555 ] Nick Dimiduk commented on HBASE-9117: - The rub is, HTableFactory uses HTablePool, which is used by thrift2 (https://github.com/ndimiduk/hbase/commit/985dbad5119348245d6d5ebff4d8a68a2d4b2263). I could probably change it to use HConnection instead (and depend on connection cache). > Remove HTablePool and all HConnection pooling related APIs > -- > > Key: HBASE-9117 > URL: https://issues.apache.org/jira/browse/HBASE-9117 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Nick Dimiduk >Priority: Critical > Fix For: 0.99.0 > > Attachments: HBASE-9117.00.patch, HBASE-9117.01.patch, > HBASE-9117.02.patch, HBASE-9117.03.patch, HBASE-9117.04.patch, > HBASE-9117.05.patch, HBASE-9117.06.patch > > > The recommended way is now: > # Create an HConnection: HConnectionManager.createConnection(...) > # Create a light HTable: HConnection.getTable(...) > # table.close() > # connection.close() > All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932537#comment-13932537 ] stack commented on HBASE-9117: -- bq. Further, if we introduce the new API as we've discussed with Enis Soztutar, 1.0 will ship with no fewer than 5 ways to acquire a handle on a cluster table ... Surely we can drop something. The pool and the factory are my preference. You have a point (smile). HTablePool has been deprecated with a while now, https://github.com/apache/hbase/commit/e4c01dac6bb9cb06a4319a5b2328163911f3aef4, since 0.96 by my reading so it should be safe to drop. > Remove HTablePool and all HConnection pooling related APIs > -- > > Key: HBASE-9117 > URL: https://issues.apache.org/jira/browse/HBASE-9117 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Nick Dimiduk >Priority: Critical > Fix For: 0.99.0 > > Attachments: HBASE-9117.00.patch, HBASE-9117.01.patch, > HBASE-9117.02.patch, HBASE-9117.03.patch, HBASE-9117.04.patch, > HBASE-9117.05.patch, HBASE-9117.06.patch > > > The recommended way is now: > # Create an HConnection: HConnectionManager.createConnection(...) > # Create a light HTable: HConnection.getTable(...) > # table.close() > # connection.close() > All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10734) Fix RegionStates.getRegionAssignments to not add duplicate regions
[ https://issues.apache.org/jira/browse/HBASE-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-10734: Attachment: 10734-1.txt Straightforward patch. > Fix RegionStates.getRegionAssignments to not add duplicate regions > -- > > Key: HBASE-10734 > URL: https://issues.apache.org/jira/browse/HBASE-10734 > Project: HBase > Issue Type: Sub-task >Reporter: Devaraj Das >Assignee: Devaraj Das > Attachments: 10734-1.txt > > > RegionStates.getRegionAssignments(Collection) erroneously adds the > default replica twice to the value list in the returned map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10733) Shell incr command should allow last parameter (value) to be missing. As documented.
[ https://issues.apache.org/jira/browse/HBASE-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932536#comment-13932536 ] Jean-Marc Spaggiari commented on HBASE-10733: - Fine in trunk, 0.98 and 0.94. Fix for 0.96 coming. > Shell incr command should allow last parameter (value) to be missing. As > documented. > > > Key: HBASE-10733 > URL: https://issues.apache.org/jira/browse/HBASE-10733 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10733) Shell incr command should allow last parameter (value) to be missing. As documented.
[ https://issues.apache.org/jira/browse/HBASE-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-10733: Affects Version/s: 0.96.1.1 > Shell incr command should allow last parameter (value) to be missing. As > documented. > > > Key: HBASE-10733 > URL: https://issues.apache.org/jira/browse/HBASE-10733 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-10734) Fix RegionStates.getRegionAssignments to not add duplicate regions
Devaraj Das created HBASE-10734: --- Summary: Fix RegionStates.getRegionAssignments to not add duplicate regions Key: HBASE-10734 URL: https://issues.apache.org/jira/browse/HBASE-10734 Project: HBase Issue Type: Sub-task Reporter: Devaraj Das Assignee: Devaraj Das RegionStates.getRegionAssignments(Collection) erroneously adds the default replica twice to the value list in the returned map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-10731:
---
Assignee: Albert Chu
Albert, I added you as an HBase contributor. Thanks!
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Assignee: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10731) Fix environment variables typos in scripts
[
https://issues.apache.org/jira/browse/HBASE-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-10731:
---
Resolution: Fixed
Fix Version/s: 0.94.18
0.99.0
0.98.1
0.96.2
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Fix environment variables typos in scripts
> --
>
> Key: HBASE-10731
> URL: https://issues.apache.org/jira/browse/HBASE-10731
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.99.0
>Reporter: Albert Chu
>Priority: Trivial
> Fix For: 0.96.2, 0.98.1, 0.99.0, 0.94.18
>
> Attachments: HBASE-10731.patch
>
>
> I noticed in the top of many of the scripts in bin/ that the old environment
> variables are documented, such as
> {noformat}
> HADOOP_SSH_OPTS Options passed to ssh when running remote commands.
> {noformat}
> However, in the script code, HBASE_SSH_OPTS is clearly used.
> I've attached a trivial script to fix this in many locations.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10620) LoadBalancer.needsBalance() should check for co-located region replicas as well
[ https://issues.apache.org/jira/browse/HBASE-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932516#comment-13932516 ] Devaraj Das commented on HBASE-10620: - Makes sense. +1 on the addendum. > LoadBalancer.needsBalance() should check for co-located region replicas as > well > --- > > Key: HBASE-10620 > URL: https://issues.apache.org/jira/browse/HBASE-10620 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10620-1.txt, 10620-2.txt, 10620-3-addendum.patch, > 10620-3.txt > > > This is a left over TODO from reviews of HBASE-10351. > LB.needsBalance() does some basic checking before running the LB.balance() > method. We need to check whether there are co-located regions in this method > so that the balancer can increase availability. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10729) Enable table doesn't balance out replicas evenly if the replicas were unassigned earlier
[ https://issues.apache.org/jira/browse/HBASE-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10729: -- Fix Version/s: hbase-10070 > Enable table doesn't balance out replicas evenly if the replicas were > unassigned earlier > > > Key: HBASE-10729 > URL: https://issues.apache.org/jira/browse/HBASE-10729 > Project: HBase > Issue Type: Sub-task >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10729-1.txt > > > Enable table doesn't assign out replicas keeping availability in mind, if the > replicas were unassigned before the table was disabled. For example, when a > snapshot is restored and then the table is enabled, the replicas are unevenly > assigned. The reason for this is that the the enable table invokes > randomAssign that assigns one region at a time. Since the master doesn't have > any information about the unassigned replicas, the calls to randomAssign > can't do any availability checks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10729) Enable table doesn't balance out replicas evenly if the replicas were unassigned earlier
[ https://issues.apache.org/jira/browse/HBASE-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932515#comment-13932515 ] Enis Soztutar commented on HBASE-10729: --- +1 > Enable table doesn't balance out replicas evenly if the replicas were > unassigned earlier > > > Key: HBASE-10729 > URL: https://issues.apache.org/jira/browse/HBASE-10729 > Project: HBase > Issue Type: Sub-task >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: hbase-10070 > > Attachments: 10729-1.txt > > > Enable table doesn't assign out replicas keeping availability in mind, if the > replicas were unassigned before the table was disabled. For example, when a > snapshot is restored and then the table is enabled, the replicas are unevenly > assigned. The reason for this is that the the enable table invokes > randomAssign that assigns one region at a time. Since the master doesn't have > any information about the unassigned replicas, the calls to randomAssign > can't do any availability checks. -- This message was sent by Atlassian JIRA (v6.2#6252)
