[jira] [Created] (HBASE-18374) RegionServer Metrics improvements
Abhishek Singh Chouhan created HBASE-18374: -- Summary: RegionServer Metrics improvements Key: HBASE-18374 URL: https://issues.apache.org/jira/browse/HBASE-18374 Project: HBase Issue Type: Improvement Affects Versions: 3.0.0 Reporter: Abhishek Singh Chouhan Assignee: Abhishek Singh Chouhan Fix For: 3.0.0 At the RS level we have latency metrics for mutate/puts and deletes that are updated per batch (ie. at the end of entire batchop if it contains put/delete update the respective metric) in contrast with append/increment/get metrics that are updated per op. This is a bit ambiguous since the delete and put metrics are updated for multi row mutations that happen to contain a put/delete. We should rename the metric(eg. delete_batch)/add better description. Also we should add metrics for single delete client operations that come through RSRpcServer.mutate path. We should also add metrics for checkAndPut and checkAndDelete. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085240#comment-16085240
]
Allan Yang commented on HBASE-18368:
{quote}
In patch v2, other than NEXT_ROW, 3 ReturnCode's are covered by the change.
Is it possible to add unit test for (some) of them ?
{quote}
2 ReturnCode's are covered by the change. the Other one INCLUDE_AND_NEXT_COL
is already covered by HBASE-17678's UT.
> Filters with OR do not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18370) Master should attempt reassignment of regions in FAILED_OPEN state
[ https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085229#comment-16085229 ] Allan Yang commented on HBASE-18370: Currently, we will retry 10 times(in default) if region failed to open, then it will set the region to FAILED_OPEN state. But meta regions will ignore the retry limit and retry forever. Adding retry number will help, too? > Master should attempt reassignment of regions in FAILED_OPEN state > -- > > Key: HBASE-18370 > URL: https://issues.apache.org/jira/browse/HBASE-18370 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell > > Currently once a region goes into FAILED_OPEN state this requires operator > intervention. With some underlying causes, this is necessary. With others, > the master could eventually successfully deploy the region without humans in > the loop. The master should optionally attempt automatic resolution of > FAILED_OPEN states with a strategy of: delay, unassign, reassign. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17442: --- Attachment: HBASE-17442.v2.patch Retry for hadoop QA. > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.v1.patch, HBASE-17442.v2.patch, HBASE-17442.v2.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[
https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085207#comment-16085207
]
Hadoop QA commented on HBASE-17442:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
4s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 8s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
52s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 21s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}143m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay |
| | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.TestRowTooBig |
| | org.apache.hadoop.hbase.mapreduce.TestWALPlayer |
| | org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| | org.apache.hadoop.hbase.regionserver.wal.TestFSHLog |
| | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.master.procedure.TestSafemodeBringsDownMaster |
| | org.apache.hadoop.hbase.backup.TestRemoteBackup |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| |
[jira] [Commented] (HBASE-18361) CellComparator compare function should compare cell's value along with other attributes
[
https://issues.apache.org/jira/browse/HBASE-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085204#comment-16085204
]
Anoop Sam John commented on HBASE-18361:
bq.we can add a new static helper method to CellUtil that compares everything.
+1 for this.. Lets not change any thing in CellComparator (IMHO)
> CellComparator compare function should compare cell's value along with other
> attributes
> ---
>
> Key: HBASE-18361
> URL: https://issues.apache.org/jira/browse/HBASE-18361
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Amit Kabra
>Assignee: Amit Kabra
> Attachments: HBASE-18361.patch
>
>
> CellComparator.compare(Cell a, Cell b) should compare a and b's value as well.
> If we create two cells as
> byte [] row = Bytes.toBytes("row");
> byte [] value = Bytes.toBytes("value");
> byte [] value1 = Bytes.toBytes("value1");
> Cell c1 = CellUtil.createCell(row, value);
> Cell c2 = CellUtil.createCell(row, value1);
> And compare them using CellComparator.compare(c1, c2,true) , it returns 0 i.e
> it matches them while they are different.CellComparator compares each
> attribute of cell but value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18361) CellComparator compare function should compare cell's value along with other attributes
[
https://issues.apache.org/jira/browse/HBASE-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085190#comment-16085190
]
Chia-Ping Tsai commented on HBASE-18361:
bq. Which of these three options is most preferable?
If Amit needs the "equal" rather than "compare", we just add the "equal"
function into CellUtil.
> CellComparator compare function should compare cell's value along with other
> attributes
> ---
>
> Key: HBASE-18361
> URL: https://issues.apache.org/jira/browse/HBASE-18361
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Amit Kabra
>Assignee: Amit Kabra
> Attachments: HBASE-18361.patch
>
>
> CellComparator.compare(Cell a, Cell b) should compare a and b's value as well.
> If we create two cells as
> byte [] row = Bytes.toBytes("row");
> byte [] value = Bytes.toBytes("value");
> byte [] value1 = Bytes.toBytes("value1");
> Cell c1 = CellUtil.createCell(row, value);
> Cell c2 = CellUtil.createCell(row, value1);
> And compare them using CellComparator.compare(c1, c2,true) , it returns 0 i.e
> it matches them while they are different.CellComparator compares each
> attribute of cell but value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085186#comment-16085186
]
Hadoop QA commented on HBASE-18368:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
21s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s{color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
18s{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 5s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
56s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 36s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestCompactionInDeadRegionServer |
| | hadoop.hbase.regionserver.TestRSKilledWhenInitializing |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Clie
[jira] [Commented] (HBASE-18365) Eliminate the findbugs warnings for hbase-common
[ https://issues.apache.org/jira/browse/HBASE-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085175#comment-16085175 ] Chia-Ping Tsai commented on HBASE-18365: [~mdrob] Thanks for the reviews. Will commit this later. > Eliminate the findbugs warnings for hbase-common > > > Key: HBASE-18365 > URL: https://issues.apache.org/jira/browse/HBASE-18365 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18365.branch-1.2.v0.patch, > HBASE-18365.branch-1.3.v0.patch, HBASE-18365.branch-1.v0.patch, > HBASE-18465.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085174#comment-16085174
]
Chia-Ping Tsai commented on HBASE-18368:
bq. It is already included in the UT, similar logic.
IMHO We should guarantee the client test reported by user works well.
> Filters with OR do not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085143#comment-16085143
]
Hadoop QA commented on HBASE-18368:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
2s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
56s{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 10s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
56s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 90m 43s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}128m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestReplicasClient |
| | hadoop.hbase.regionserver.TestRSKilledWhenInitializing |
| | hadoop.hbase.client.TestSnapshotWithAcl |
\\
\\
|| Subsystem || Repo
[jira] [Commented] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085133#comment-16085133
]
Ted Yu commented on HBASE-18368:
In patch v2, other than NEXT_ROW, 3 ReturnCode's are covered by the change.
Is it possible to add unit test for (some) of them ?
> Filters with OR do not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085111#comment-16085111
]
Allan Yang commented on HBASE-18368:
{quote}
Would you please add the test writed by peter ?
{quote}
It is already included in the UT, similar logic.
> Filters with OR do not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[
https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085110#comment-16085110
]
Hadoop QA commented on HBASE-18175:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hbase-spark-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
13s{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}132m 32s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}189m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestServerCrashProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18175 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876984/hbase-18175-master-v9.patch
|
| Optional Tests | asflicense javac javadoc unit xml compile findbugs
h
[jira] [Commented] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085092#comment-16085092
]
Chia-Ping Tsai commented on HBASE-18368:
Would you please add the test writed by peter ?
> Filters with OR do not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17442: --- Attachment: HBASE-17442.v2.patch > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0 > > Attachments: 0001-hbase-replication-module.patch, > HBASE-17442.v1.patch, HBASE-17442.v2.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18368) Filters with OR do not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18368:
---
Summary: Filters with OR do not work (was: FamilyFilters with OR does not
work)
> Filters with OR do not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17678) FilterList with MUST_PASS_ONE may lead to redundant cells returned
[
https://issues.apache.org/jira/browse/HBASE-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085082#comment-16085082
]
Chia-Ping Tsai commented on HBASE-17678:
HBASE-18368 will fix the code for this jira so I think we can push this right
now.
> FilterList with MUST_PASS_ONE may lead to redundant cells returned
> --
>
> Key: HBASE-17678
> URL: https://issues.apache.org/jira/browse/HBASE-17678
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 1.3.0, 1.2.1
> Environment: RedHat 7.x
>Reporter: Jason Tokayer
>Assignee: Zheng Hu
> Attachments: HBASE-17678.addendum.patch, HBASE-17678.addendum.patch,
> HBASE-17678.branch-1.1.v1.patch, HBASE-17678.branch-1.1.v2.patch,
> HBASE-17678.branch-1.1.v2.patch, HBASE-17678.branch-1.1.v2.patch,
> HBASE-17678.branch-1.v1.patch, HBASE-17678.branch-1.v1.patch,
> HBASE-17678.branch-1.v2.patch, HBASE-17678.branch-1.v2.patch,
> HBASE-17678.v1.patch, HBASE-17678.v1.rough.patch, HBASE-17678.v2.patch,
> HBASE-17678.v3.patch, HBASE-17678.v4.patch, HBASE-17678.v4.patch,
> HBASE-17678.v5.patch, HBASE-17678.v6.patch, HBASE-17678.v7.patch,
> HBASE-17678.v7.patch, TestColumnPaginationFilterDemo.java
>
>
> When combining ColumnPaginationFilter with a single-element filterList,
> MUST_PASS_ONE and MUST_PASS_ALL give different results when there are
> multiple cells with the same timestamp. This is unexpected since there is
> only a single filter in the list, and I would believe that MUST_PASS_ALL and
> MUST_PASS_ONE should only affect the behavior of the joined filter and not
> the behavior of any one of the individual filters. If this is not a bug then
> it would be nice if the documentation is updated to explain this nuanced
> behavior.
> I know that there was a decision made in an earlier Hbase version to keep
> multiple cells with the same timestamp. This is generally fine but presents
> an issue when using the aforementioned filter combination.
> Steps to reproduce:
> In the shell create a table and insert some data:
> {code:none}
> create 'ns:tbl',{NAME => 'family',VERSIONS => 100}
> put 'ns:tbl','row','family:name','John',1
> put 'ns:tbl','row','family:name','Jane',1
> put 'ns:tbl','row','family:name','Gil',1
> put 'ns:tbl','row','family:name','Jane',1
> {code}
> Then, use a Scala client as:
> {code:none}
> import org.apache.hadoop.hbase.filter._
> import org.apache.hadoop.hbase.util.Bytes
> import org.apache.hadoop.hbase.client._
> import org.apache.hadoop.hbase.{CellUtil, HBaseConfiguration, TableName}
> import scala.collection.mutable._
> val config = HBaseConfiguration.create()
> config.set("hbase.zookeeper.quorum", "localhost")
> config.set("hbase.zookeeper.property.clientPort", "2181")
> val connection = ConnectionFactory.createConnection(config)
> val logicalOp = FilterList.Operator.MUST_PASS_ONE
> val limit = 1
> var resultsList = ListBuffer[String]()
> for (offset <- 0 to 20 by limit) {
> val table = connection.getTable(TableName.valueOf("ns:tbl"))
> val paginationFilter = new ColumnPaginationFilter(limit,offset)
> val filterList: FilterList = new FilterList(logicalOp,paginationFilter)
> println("@ filterList = "+filterList)
> val results = table.get(new
> Get(Bytes.toBytes("row")).setFilter(filterList))
> val cells = results.rawCells()
> if (cells != null) {
> for (cell <- cells) {
> val value = new String(CellUtil.cloneValue(cell))
> val qualifier = new String(CellUtil.cloneQualifier(cell))
> val family = new String(CellUtil.cloneFamily(cell))
> val result = "OFFSET = "+offset+":"+family + "," + qualifier
> + "," + value + "," + cell.getTimestamp()
> resultsList.append(result)
> }
> }
> }
> resultsList.foreach(println)
> {code}
> Here are the results for different limit and logicalOp settings:
> {code:none}
> Limit = 1 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 1 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 1:family,name,Gil,1
> OFFSET = 2:family,name,Jane,1
> OFFSET = 3:family,name,John,1
> Limit = 2 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 2 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 2:family,name,Jane,1
> {code}
> So, it seems that MUST_PASS_ALL gives the expected behavior, but
> MUST_PASS_ONE does
[jira] [Comment Edited] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085068#comment-16085068
]
Allan Yang edited comment on HBASE-18368 at 7/13/17 2:42 AM:
-
{quote}
NEXT_ROW can work is because there is regionscanner and store scanner, next_row
affect store scanner.
When one storescanner switch to next row, the region scanner switch to the next
storescanner, the second store scanner will continue check the current row.
That why the change can be more efficiency.
{quote}
It makes sense, we shouldn't revert this optimization. The problem is that
filters are only be reset after row change. HBASE-17678 introduce a list to
store the previous code and cell, but it does not consider the family doesn't
match case(change to another storescanner). update a V2 patch to fix this.
was (Author: allan163):
{quote}
NEXT_ROW can work is because there is regionscanner and store scanner, next_row
affect store scanner.
When one storescanner switch to next row, the region scanner switch to the next
storescanner, the second store scanner will continue check the current row.
That why the change can be more efficiency.
{quote}
It make sense, we shouldn't revert this optimization. The problem is that
filters are only be reset after row change. HBASE-17678 introduce a list to
store the previous code and cell, but it does not consider the family doesn't
match case. update a V2 patch to fix this.
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-18368:
---
Attachment: HBASE-18368.branch-1.v2.patch
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085068#comment-16085068
]
Allan Yang commented on HBASE-18368:
{quote}
NEXT_ROW can work is because there is regionscanner and store scanner, next_row
affect store scanner.
When one storescanner switch to next row, the region scanner switch to the next
storescanner, the second store scanner will continue check the current row.
That why the change can be more efficiency.
{quote}
It make sense, we shouldn't revert this optimization. The problem is that
filters are only be reset after row change. HBASE-17678 introduce a list to
store the previous code and cell, but it does not consider the family doesn't
match case. update a V2 patch to fix this.
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch, HBASE-18368.branch-1.v2.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18373) fstat unimplemented error on Rasbian Jessie
Nathan Green created HBASE-18373: Summary: fstat unimplemented error on Rasbian Jessie Key: HBASE-18373 URL: https://issues.apache.org/jira/browse/HBASE-18373 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.0.0-alpha-1 Environment: Raspberry Pi 3 Rasbian Jessie (latest version) Reporter: Nathan Green Priority: Blocker I have a Hbase distributed cluster setup on my Raspberry PI cluster (1 x Namenode & 4 x Datanode) The stable version of Hbase shell works fine... unfortunately I had hadoop issues (mismatch of jar files gr) so I have installed latest Hbase & installed matching version of Hadoop (2.7.1). HBase itself is up and running and all good. Hbase shell however gives this error: pi@pidoop1:/opt/hbase/lib/ruby/irb $ hbase shell 2017-07-13 12:25:11,722 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] HBase Shell Use "help" to get list of supported commands. Use "exit" to quit this interactive shell. Version 2.0.0-alpha-1, rc830a0f47f58d4892dd3300032c8244d6278aecc, Wed Jun 7 22:05:17 PDT 2017 Took 0.0300 seconds NotImplementedError: fstat unimplemented unsupported or native support failed to load; see http://wiki.jruby.org/Native-Libraries initialize at org/jruby/RubyIO.java:1013 open at org/jruby/RubyIO.java:1154 initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141 initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70 initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:426 initialize at /opt/hbase/lib/ruby/irb/hirb.rb:43 start at /opt/hbase/bin/hirb.rb:194 at /opt/hbase/bin/hirb.rb:206 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-18368:
---
Attachment: (was: HBASE-18368.branch-1.patch)
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-18368:
---
Attachment: HBASE-18368.branch-1.patch
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18202) Trim down supplemental models file for unnecessary entries
[ https://issues.apache.org/jira/browse/HBASE-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084985#comment-16084985 ] Hudson commented on HBASE-18202: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3364 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3364/]) HBASE-18202 Remove unnecessary info from supplemental-models (busbey: rev 22df926541a2427d07f4a39e92c26069a889bc3f) * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Trim down supplemental models file for unnecessary entries > -- > > Key: HBASE-18202 > URL: https://issues.apache.org/jira/browse/HBASE-18202 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18202.patch, HBASE-18202.v2.patch > > > With the more permissive "Apache License" check in HBASE-18033, we can remove > many entries from the supplemental-models.xml file. This issue is to track > that work separately. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18372) Potential infinite busy loop in HMaster's ProcedureExecutor
Benoit Sigoure created HBASE-18372:
--
Summary: Potential infinite busy loop in HMaster's
ProcedureExecutor
Key: HBASE-18372
URL: https://issues.apache.org/jira/browse/HBASE-18372
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 1.3.1
Environment: Kernel 3.10.0-327.10.1.el7.x86_64
JVM 1.8.0_102
Reporter: Benoit Sigoure
While investigating an issue today with [~timoha] we saw the HMaster
consistently burning 1.5 cores of CPU cycles. Upon looking more closely, it
was actually all 8 threads of {{ProcedureExecutor}} thread pool taking
constantly ~15% of a CPU core each (I identified this by looking at individual
threads in {{top}} and cross-referencing the thread IDs with the thread IDs in
a JVM stack trace). The HMaster log or output didn't contain anything
suspicious and it was hard for us to ascertain what exactly was happening. It
just looked like these threads were regularly spinning, doing nothing. We just
saw a lot of {{futex}} system calls happening all the time, and all the threads
of the thread pool regularly taking turns in waking up and going back to sleep.
My reading of the code in {{procedure2/ProcedureExecutor.java}} is that this
can happen if the threads in the thread pool have been interrupted for some
reason:
{code}
private void execLoop() {
while (isRunning()) {
Procedure proc = runnables.poll();
if (proc == null) continue;
{code}
and then in {master/procedure/MasterProcedureScheduler.java}:
{code}
@Override
public Procedure poll() {
return poll(-1);
}
@edu.umd.cs.findbugs.annotations.SuppressWarnings("WA_AWAIT_NOT_IN_LOOP")
Procedure poll(long waitNsec) {
Procedure pollResult = null;
schedLock.lock();
try {
if (queueSize == 0) {
if (waitNsec < 0) {
schedWaitCond.await();
[...]
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
schedLock.unlock();
}
return pollResult;
}
{code}
so my theory is the threads in the thread pool have all been interrupted (maybe
by a procedure that ran earlier and left its thread interrupted) and so we are
perpetually looping in {{execLoop}}, which ends up calling
{{schedWaitCond.await();}}, which ends up throwing an {{InterruptedException}},
which ends up resetting the interrupt status of the thread, and rinse and
repeat.
But again I wasn't able to get any cold hard evidence that this is what was
happening. There was just no other evidence that could explain this behavior,
and I wasn't able to guess what else could be causing this that was consistent
with what we saw and what I understood from reading the code.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18175) Add hbase-spark integration test into hbase-spark-it
[ https://issues.apache.org/jira/browse/HBASE-18175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18175: - Attachment: hbase-18175-master-v9.patch V9 Carry Mike's Comment. Thanks Mike spending time for reviewing :). > Add hbase-spark integration test into hbase-spark-it > > > Key: HBASE-18175 > URL: https://issues.apache.org/jira/browse/HBASE-18175 > Project: HBase > Issue Type: Test > Components: spark >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0 > > Attachments: hbase-18175-master-v2.patch, > hbase-18175-master-v3.patch, hbase-18175-master-v4.patch, > hbase-18175-master-v5.patch, hbase-18175-master-v6.patch, > hbase-18175-master-v7.patch, hbase-18175-master-v8.patch, > hbase-18175-master-v9.patch, hbase-18175-v1.patch > > > After HBASE-17574, all test under hbase-spark are regarded as unit test, and > this jira will add integration test of hbase-spark into hbase-it. This patch > run same tests as mapreduce.IntegrationTestBulkLoad, just change mapreduce to > spark. > test in Maven: > mvn verify -Dit.test=IntegrationTestSparkBulkLoad > test on cluster: > spark-submit --class > org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad > HBASE_HOME/lib/hbase-it-2.0.0-SNAPSHOT-tests.jar > -Dhbase.spark.bulkload.chainlength=50 -m slowDeterministic -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18229) create new Async Split API to embrace AM v2
[ https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084852#comment-16084852 ] Yi Liang commented on HBASE-18229: -- hmm, no unit test error? wired. > create new Async Split API to embrace AM v2 > --- > > Key: HBASE-18229 > URL: https://issues.apache.org/jira/browse/HBASE-18229 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0-alpha-2 > > Attachments: HBase-18229-master-v1.patch, > hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch, > hbase-18229-master-v4.patch, hbase-18229-master-v5.patch, > hbase-18229-master-v6.patch, hbase-18229-master-v7.patch, > hbase-18229-master-v7.patch, hbase-18229-master-v8.patch > > > See HBASE-18105 for related information, this jira will change the logic of > Path 1 in splitProcedure, the execution path will be: > *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> > MasterRpcServices.splitRegion -> HMaster.splitRegion-> > AssignmentManager.submitProcedure* > Master Will no longer ask Rs and then Rs turn around to ask master to do the > split operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18371) Update folly and wangle dependencies
Enis Soztutar created HBASE-18371: - Summary: Update folly and wangle dependencies Key: HBASE-18371 URL: https://issues.apache.org/jira/browse/HBASE-18371 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: HBASE-14850 We need to update folly and wangle dependency versions. Debugging an issue, I realized that we may need a couple of recent patches from wangle. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18338) [C++] Implement RpcTestServer
[ https://issues.apache.org/jira/browse/HBASE-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084771#comment-16084771 ] Xiaobing Zhou edited comment on HBASE-18338 at 7/12/17 9:50 PM: Posted v2: # being able to do ping/pong # removed Bonk and Xtruct # fixed all compile issues # changed serde/rpc.h&rpc.cc to serde/rpc-serde.h&rpc-serde.cc for less confusion. Will port protobuf types into RPC test client/server by protobuf-compiling test.proto and test_rpc_service.proto. was (Author: xiaobingo): Posted v2: # being able to do ping/pong # removed Bonk and Xtruct # fixed all compile issues Will port protobuf types into RPC test client/server by protobuf-compiling test.proto and test_rpc_service.proto. > [C++] Implement RpcTestServer > - > > Key: HBASE-18338 > URL: https://issues.apache.org/jira/browse/HBASE-18338 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18338.000.patch, HBASE-18338.001.patch, > HBASE-18338.002.patch > > > This is a spin-off from HBASE-18078. We need RpcTestServer to simulate > various communication scenarios, e.g. timeout, connection aborted, long > running services and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18260) Address new license dependencies from hadoop3-alpha4
[ https://issues.apache.org/jira/browse/HBASE-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084833#comment-16084833 ] Hudson commented on HBASE-18260: FAILURE: Integrated in Jenkins build HBase-2.0 #162 (See [https://builds.apache.org/job/HBase-2.0/162/]) HBASE-18260 h3-a4 license additions (busbey: rev 6a891587afa4f0808d66c831441661216cb06794) * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Address new license dependencies from hadoop3-alpha4 > > > Key: HBASE-18260 > URL: https://issues.apache.org/jira/browse/HBASE-18260 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18260.patch > > > hadoop3-alpha4 adds at least one new dependency that requires an update to > our supplemental-models.xml > use this jira to track the changes for after that is released. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084805#comment-16084805
]
Hadoop QA commented on HBASE-18368:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-18368 does not apply to branch-1. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18368 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876885/HBASE-18368.branch-1.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7641/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18357) Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084823#comment-16084823 ] Vladimir Rodionov edited comment on HBASE-18357 at 7/12/17 10:16 PM: - The test testClusterStatus fails because client *ConnectionImplementation* does not receive ClusterStatus periodic notifications - there is no code at all. It seems that server side publisher is in place and has Multicast implementation, so we need implement multicast *ClusterStatusListener* cc: [~syuanjiang] was (Author: vrodionov): The test testClusterStatus fails because client *ConnectionImplementation* does not receive ClusterStatus periodic notifications - there is no code at all. It seems that server side publisher is in place and has Multicast implementation, so we need implement multicast ClusterStatusListener. cc: [~syuanjiang] > Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in > HBASE-14614 > > > Key: HBASE-18357 > URL: https://issues.apache.org/jira/browse/HBASE-18357 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > > The Core Proc-V2 AM change in HBASE-14614 disabled two tests inTestHCM: > testMulti and testRegionCaching > This JIRA tracks the work to enable them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18371) [C++] Update folly and wangle dependencies
[ https://issues.apache.org/jira/browse/HBASE-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18371: -- Attachment: hbase-18371_v1.patch v1 patch. The retry test is hanging though. > [C++] Update folly and wangle dependencies > -- > > Key: HBASE-18371 > URL: https://issues.apache.org/jira/browse/HBASE-18371 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: HBASE-14850 > > Attachments: hbase-18371_v1.patch > > > We need to update folly and wangle dependency versions. Debugging an issue, I > realized that we may need a couple of recent patches from wangle. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18357) Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084823#comment-16084823 ] Vladimir Rodionov commented on HBASE-18357: --- The test testClusterStatus fails because client *ConnectionImplementation* does not receive ClusterStatus periodic notifications - there is no code at all. It seems that server side publisher is in place and has Multicast implementation, so we need implement multicast ClusterStatusListener. cc: [~syuanjiang] > Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in > HBASE-14614 > > > Key: HBASE-18357 > URL: https://issues.apache.org/jira/browse/HBASE-18357 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > > The Core Proc-V2 AM change in HBASE-14614 disabled two tests inTestHCM: > testMulti and testRegionCaching > This JIRA tracks the work to enable them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18371) [C++] Update folly and wangle dependencies
[ https://issues.apache.org/jira/browse/HBASE-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18371: -- Summary: [C++] Update folly and wangle dependencies (was: Update folly and wangle dependencies) > [C++] Update folly and wangle dependencies > -- > > Key: HBASE-18371 > URL: https://issues.apache.org/jira/browse/HBASE-18371 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: HBASE-14850 > > > We need to update folly and wangle dependency versions. Debugging an issue, I > realized that we may need a couple of recent patches from wangle. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084785#comment-16084785 ] Andrew Purtell commented on HBASE-18095: bq. Should the new configuration setting for the master locations be set in the conf/hbase-site.xml under a new property? Yes > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Purtell >Assignee: Ali > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18338) [C++] Implement RpcTestServer
[ https://issues.apache.org/jira/browse/HBASE-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084771#comment-16084771 ] Xiaobing Zhou commented on HBASE-18338: --- Posted v2: # being able to do ping/pong # removed Bonk and Xtruct # fixed all compile issues Will port protobuf types into RPC test client/server by protobuf-compiling test.proto and test_rpc_service.proto. > [C++] Implement RpcTestServer > - > > Key: HBASE-18338 > URL: https://issues.apache.org/jira/browse/HBASE-18338 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18338.000.patch, HBASE-18338.001.patch, > HBASE-18338.002.patch > > > This is a spin-off from HBASE-18078. We need RpcTestServer to simulate > various communication scenarios, e.g. timeout, connection aborted, long > running services and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18338) [C++] Implement RpcTestServer
[ https://issues.apache.org/jira/browse/HBASE-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18338: -- Attachment: HBASE-18338.002.patch > [C++] Implement RpcTestServer > - > > Key: HBASE-18338 > URL: https://issues.apache.org/jira/browse/HBASE-18338 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18338.000.patch, HBASE-18338.001.patch, > HBASE-18338.002.patch > > > This is a spin-off from HBASE-18078. We need RpcTestServer to simulate > various communication scenarios, e.g. timeout, connection aborted, long > running services and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18229) create new Async Split API to embrace AM v2
[
https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084757#comment-16084757
]
Hadoop QA commented on HBASE-18229:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m
0s{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m
0s{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 16m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
18s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
12s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 16m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 32s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
47s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}116m
59s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
3s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
0s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {c
[jira] [Commented] (HBASE-7386) Investigate providing some supervisor support for znode deletion
[ https://issues.apache.org/jira/browse/HBASE-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084730#comment-16084730 ] Samir Ahmic commented on HBASE-7386: [~stack] i have done some testing with last patches against master branch and good news is that most of code(with small changes) and functionality works fine. So original idea to improve MTTR by removing stale master and rs znodes plus watchdog which will restart process in case of unexpected failure is still valid. My original scripts here are written with idea to be optional route in managing hbase processes using supervisor, and that approach opens couple of questions which i would like to discuss: # Amount of code added and options to reduce it (i will anyway try to reduce it to minimum) probably some code can be integrated in exiting scripts to avoid copying # Where are we going to add new scripts supervisord folder inside bin dir was may original idea and same thing goes for config files supervisord folder in conf dir # Testing: i will cover supervisor 3.3.2 version(last stable) and some older version that are installed trough system packet manages # And finally would it be better to implement our own Java supervisor which would do similar thing as python implementation Based on what we decide i will continue work here, if we go with python supervisor i can have patch ready for testing in couple of days. > Investigate providing some supervisor support for znode deletion > > > Key: HBASE-7386 > URL: https://issues.apache.org/jira/browse/HBASE-7386 > Project: HBase > Issue Type: Task > Components: master, regionserver, scripts >Reporter: Gregory Chanan >Assignee: stack >Priority: Blocker > Attachments: HBASE-7386-bin.patch, HBASE-7386-bin-v2.patch, > HBASE-7386-bin-v3.patch, HBASE-7386-conf.patch, HBASE-7386-conf-v2.patch, > HBASE-7386-conf-v3.patch, HBASE-7386-src.patch, HBASE-7386-v0.patch, > supervisordconfigs-v0.patch > > > There a couple of JIRAs for deleting the znode on a process failure: > HBASE-5844 (RS) > HBASE-5926 (Master) > which are pretty neat; on process failure, they delete the znode of the > underlying process so HBase can recover faster. > These JIRAs were implemented via the startup scripts; i.e. the script hangs > around and waits for the process to exit, then deletes the znode. > There are a few problems associated with this approach, as listed in the > below JIRAs: > 1) Hides startup output in script > https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463401&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463401 > 2) two hbase processes listed per launched daemon > https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463409&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463409 > 3) Not run by a real supervisor > https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463409&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463409 > 4) Weird output after kill -9 actual process in standalone mode > https://issues.apache.org/jira/browse/HBASE-5926?focusedCommentId=13506801&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13506801 > 5) Can kill existing RS if called again > https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463401&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463401 > 6) Hides stdout/stderr[6] > https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13506832&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13506832 > I suspect running in via something like supervisor.d can solve these issues > if we provide the right support. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18061) [C++] Fix retry logic in multi-get calls
[ https://issues.apache.org/jira/browse/HBASE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084716#comment-16084716 ] Sudeep Sunthankar commented on HBASE-18061: --- Thanks [~tedyu], yes I can see there are some issues. I have rectified the same and testing the patch. > [C++] Fix retry logic in multi-get calls > > > Key: HBASE-18061 > URL: https://issues.apache.org/jira/browse/HBASE-18061 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Sudeep Sunthankar > Fix For: HBASE-14850 > > Attachments: HBASE-18061.HBASE-14850.v1.patch, > HBASE-18061.HBASE-14850.v3.patch, HBASE-18061.HBASE-14850.v5.patch, > HBASE-18061.HBASE-14850.v6.patch, HBASE-18061.HBASE-14850.v7.patch > > > HBASE-17576 adds multi-gets. There are a couple of todos to fix in the retry > logic, and some unit testing to be done for the multi-gets. We'll do these in > this issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18339) Update test-patch to use hadoop 3.0.0-alpha4
[
https://issues.apache.org/jira/browse/HBASE-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084721#comment-16084721
]
Hadoop QA commented on HBASE-18339:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m
49s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
6s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 33s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 87m 23s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
51s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.master.TestMasterMetricsWrapper |
| | org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
| | org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay |
| | org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | org.apache.hadoop.hbase.regionserver.wal.TestAsyncWALReplay |
| | org.apache.hadoop.hbase.regionserver.TestRowTooBig |
| | org.apache.hadoop.hbase.mapreduce.TestTableInputFormat |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.trace.TestHTraceHooks |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | org.apache.hadoop.hbase.regionserver.TestRemoveRegionMetrics |
| | org.apache.hadoop.hbase.TestFullLogReconstruction |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics |
| | org.apache.hadoop.hbase.master.assignment.TestAssignmentManager |
| | org.apache.hadoop.hbase.mapred.TestTableInputFormat |
| | org.apache.had
[jira] [Commented] (HBASE-18364) Downgrade surefire
[ https://issues.apache.org/jira/browse/HBASE-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084661#comment-16084661 ] Hudson commented on HBASE-18364: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3363 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3363/]) HBASE-18364 Downgrade surefire to 2.19.1 because it broke package (elserj: rev 1800580a8ad313ffeb185d3110e426bde027f180) * (edit) pom.xml > Downgrade surefire > -- > > Key: HBASE-18364 > URL: https://issues.apache.org/jira/browse/HBASE-18364 > Project: HBase > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-18364.001.patch > > > Seems like 2.20 broke package-wildcard for test specification. Pull this back > to something that works. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18260) Address new license dependencies from hadoop3-alpha4
[ https://issues.apache.org/jira/browse/HBASE-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084662#comment-16084662 ] Hudson commented on HBASE-18260: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3363 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3363/]) HBASE-18260 h3-a4 license additions (busbey: rev 8623d14632b1e95513616f933d9e0150b27648fd) * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > Address new license dependencies from hadoop3-alpha4 > > > Key: HBASE-18260 > URL: https://issues.apache.org/jira/browse/HBASE-18260 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18260.patch > > > hadoop3-alpha4 adds at least one new dependency that requires an update to > our supplemental-models.xml > use this jira to track the changes for after that is released. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084653#comment-16084653 ] Ali commented on HBASE-18095: - Should the new configuration setting for the master locations be set in the conf/hbase-site.xml under a new property? > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Purtell >Assignee: Ali > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18061) [C++] Fix retry logic in multi-get calls
[
https://issues.apache.org/jira/browse/HBASE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084639#comment-16084639
]
Ted Yu commented on HBASE-18061:
After resolving the above conflict, I got the following compilation errors:
{code}
core/response-converter.cc:154:55: error: no viable conversion from
'shared_ptr' to 'shared_ptr'
multi_response->AddRegionException(region_name, ew);
^~
core/response-converter.cc:174:57: error: no viable conversion from
'shared_ptr' to 'shared_ptr'
multi_response->AddRegionException(region_name, ew);
^~
{code}
> [C++] Fix retry logic in multi-get calls
>
>
> Key: HBASE-18061
> URL: https://issues.apache.org/jira/browse/HBASE-18061
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Sudeep Sunthankar
> Fix For: HBASE-14850
>
> Attachments: HBASE-18061.HBASE-14850.v1.patch,
> HBASE-18061.HBASE-14850.v3.patch, HBASE-18061.HBASE-14850.v5.patch,
> HBASE-18061.HBASE-14850.v6.patch, HBASE-18061.HBASE-14850.v7.patch
>
>
> HBASE-17576 adds multi-gets. There are a couple of todos to fix in the retry
> logic, and some unit testing to be done for the multi-gets. We'll do these in
> this issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084632#comment-16084632
]
Hadoop QA commented on HBASE-18368:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-18368 does not apply to branch-1. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18368 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876885/HBASE-18368.branch-1.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7640/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18368:
---
Status: Patch Available (was: Open)
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0-alpha-1, 3.0.0
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18361) CellComparator compare function should compare cell's value along with other attributes
[
https://issues.apache.org/jira/browse/HBASE-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084622#comment-16084622
]
Andrew Purtell commented on HBASE-18361:
There is Amit's patch.
Or we can add a new comparator that checks every part of the cell including
value.
Or we can add a new static helper method to CellUtil that compares everything.
Which of these three options is most preferable?
> CellComparator compare function should compare cell's value along with other
> attributes
> ---
>
> Key: HBASE-18361
> URL: https://issues.apache.org/jira/browse/HBASE-18361
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Amit Kabra
>Assignee: Amit Kabra
> Attachments: HBASE-18361.patch
>
>
> CellComparator.compare(Cell a, Cell b) should compare a and b's value as well.
> If we create two cells as
> byte [] row = Bytes.toBytes("row");
> byte [] value = Bytes.toBytes("value");
> byte [] value1 = Bytes.toBytes("value1");
> Cell c1 = CellUtil.createCell(row, value);
> Cell c2 = CellUtil.createCell(row, value1);
> And compare them using CellComparator.compare(c1, c2,true) , it returns 0 i.e
> it matches them while they are different.CellComparator compares each
> attribute of cell but value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18202) Trim down supplemental models file for unnecessary entries
[ https://issues.apache.org/jira/browse/HBASE-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084608#comment-16084608 ] Mike Drob commented on HBASE-18202: --- i think i did my testing on branch-2, let me double check > Trim down supplemental models file for unnecessary entries > -- > > Key: HBASE-18202 > URL: https://issues.apache.org/jira/browse/HBASE-18202 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18202.patch, HBASE-18202.v2.patch > > > With the more permissive "Apache License" check in HBASE-18033, we can remove > many entries from the supplemental-models.xml file. This issue is to track > that work separately. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18364) Downgrade surefire
[ https://issues.apache.org/jira/browse/HBASE-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084607#comment-16084607 ] Hudson commented on HBASE-18364: FAILURE: Integrated in Jenkins build HBase-2.0 #161 (See [https://builds.apache.org/job/HBase-2.0/161/]) HBASE-18364 Downgrade surefire to 2.19.1 because it broke package (elserj: rev 0eb2c3fc31fc46db4a49851957f9e255e50efb0a) * (edit) pom.xml > Downgrade surefire > -- > > Key: HBASE-18364 > URL: https://issues.apache.org/jira/browse/HBASE-18364 > Project: HBase > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-18364.001.patch > > > Seems like 2.20 broke package-wildcard for test specification. Pull this back > to something that works. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18202) Trim down supplemental models file for unnecessary entries
[ https://issues.apache.org/jira/browse/HBASE-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084593#comment-16084593 ] Sean Busbey commented on HBASE-18202: - yeah let me push it now. just master, or prefer this on some older branches? > Trim down supplemental models file for unnecessary entries > -- > > Key: HBASE-18202 > URL: https://issues.apache.org/jira/browse/HBASE-18202 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18202.patch, HBASE-18202.v2.patch > > > With the more permissive "Apache License" check in HBASE-18033, we can remove > many entries from the supplemental-models.xml file. This issue is to track > that work separately. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18202) Trim down supplemental models file for unnecessary entries
[ https://issues.apache.org/jira/browse/HBASE-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084573#comment-16084573 ] Mike Drob commented on HBASE-18202: --- Two +1s, good to commit? [~busbey] [~stack] > Trim down supplemental models file for unnecessary entries > -- > > Key: HBASE-18202 > URL: https://issues.apache.org/jira/browse/HBASE-18202 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18202.patch, HBASE-18202.v2.patch > > > With the more permissive "Apache License" check in HBASE-18033, we can remove > many entries from the supplemental-models.xml file. This issue is to track > that work separately. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17678) FilterList with MUST_PASS_ONE may lead to redundant cells returned
[
https://issues.apache.org/jira/browse/HBASE-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084541#comment-16084541
]
Sean Busbey commented on HBASE-17678:
-
Totally forgot that I was going to push these to maintenance branches.
[~chia7712] shall we wait to push until HBASE-18368 settles out?
> FilterList with MUST_PASS_ONE may lead to redundant cells returned
> --
>
> Key: HBASE-17678
> URL: https://issues.apache.org/jira/browse/HBASE-17678
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 1.3.0, 1.2.1
> Environment: RedHat 7.x
>Reporter: Jason Tokayer
>Assignee: Zheng Hu
> Attachments: HBASE-17678.addendum.patch, HBASE-17678.addendum.patch,
> HBASE-17678.branch-1.1.v1.patch, HBASE-17678.branch-1.1.v2.patch,
> HBASE-17678.branch-1.1.v2.patch, HBASE-17678.branch-1.1.v2.patch,
> HBASE-17678.branch-1.v1.patch, HBASE-17678.branch-1.v1.patch,
> HBASE-17678.branch-1.v2.patch, HBASE-17678.branch-1.v2.patch,
> HBASE-17678.v1.patch, HBASE-17678.v1.rough.patch, HBASE-17678.v2.patch,
> HBASE-17678.v3.patch, HBASE-17678.v4.patch, HBASE-17678.v4.patch,
> HBASE-17678.v5.patch, HBASE-17678.v6.patch, HBASE-17678.v7.patch,
> HBASE-17678.v7.patch, TestColumnPaginationFilterDemo.java
>
>
> When combining ColumnPaginationFilter with a single-element filterList,
> MUST_PASS_ONE and MUST_PASS_ALL give different results when there are
> multiple cells with the same timestamp. This is unexpected since there is
> only a single filter in the list, and I would believe that MUST_PASS_ALL and
> MUST_PASS_ONE should only affect the behavior of the joined filter and not
> the behavior of any one of the individual filters. If this is not a bug then
> it would be nice if the documentation is updated to explain this nuanced
> behavior.
> I know that there was a decision made in an earlier Hbase version to keep
> multiple cells with the same timestamp. This is generally fine but presents
> an issue when using the aforementioned filter combination.
> Steps to reproduce:
> In the shell create a table and insert some data:
> {code:none}
> create 'ns:tbl',{NAME => 'family',VERSIONS => 100}
> put 'ns:tbl','row','family:name','John',1
> put 'ns:tbl','row','family:name','Jane',1
> put 'ns:tbl','row','family:name','Gil',1
> put 'ns:tbl','row','family:name','Jane',1
> {code}
> Then, use a Scala client as:
> {code:none}
> import org.apache.hadoop.hbase.filter._
> import org.apache.hadoop.hbase.util.Bytes
> import org.apache.hadoop.hbase.client._
> import org.apache.hadoop.hbase.{CellUtil, HBaseConfiguration, TableName}
> import scala.collection.mutable._
> val config = HBaseConfiguration.create()
> config.set("hbase.zookeeper.quorum", "localhost")
> config.set("hbase.zookeeper.property.clientPort", "2181")
> val connection = ConnectionFactory.createConnection(config)
> val logicalOp = FilterList.Operator.MUST_PASS_ONE
> val limit = 1
> var resultsList = ListBuffer[String]()
> for (offset <- 0 to 20 by limit) {
> val table = connection.getTable(TableName.valueOf("ns:tbl"))
> val paginationFilter = new ColumnPaginationFilter(limit,offset)
> val filterList: FilterList = new FilterList(logicalOp,paginationFilter)
> println("@ filterList = "+filterList)
> val results = table.get(new
> Get(Bytes.toBytes("row")).setFilter(filterList))
> val cells = results.rawCells()
> if (cells != null) {
> for (cell <- cells) {
> val value = new String(CellUtil.cloneValue(cell))
> val qualifier = new String(CellUtil.cloneQualifier(cell))
> val family = new String(CellUtil.cloneFamily(cell))
> val result = "OFFSET = "+offset+":"+family + "," + qualifier
> + "," + value + "," + cell.getTimestamp()
> resultsList.append(result)
> }
> }
> }
> resultsList.foreach(println)
> {code}
> Here are the results for different limit and logicalOp settings:
> {code:none}
> Limit = 1 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 1 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 1:family,name,Gil,1
> OFFSET = 2:family,name,Jane,1
> OFFSET = 3:family,name,John,1
> Limit = 2 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 2 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 2:family,name,Jane,1
> {code}
> So, it seems that MUST_PASS_ALL gives t
[jira] [Commented] (HBASE-18344) Replace Append#add(byte[], byte[], byte[]) by Append#addColumn
[ https://issues.apache.org/jira/browse/HBASE-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084540#comment-16084540 ] Jan Hentschel commented on HBASE-18344: --- The build failure seems to be unrelated. > Replace Append#add(byte[], byte[], byte[]) by Append#addColumn > -- > > Key: HBASE-18344 > URL: https://issues.apache.org/jira/browse/HBASE-18344 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Jan Hentschel >Priority: Trivial > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18344.master.001.patch, > HBASE-18344.master.002.patch > > > We have Put#addColumn and Increment#addColumn but there is no > Append#addColumn. We should add Append#addColumn for consistency. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18365) Eliminate the findbugs warnings for hbase-common
[ https://issues.apache.org/jira/browse/HBASE-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084542#comment-16084542 ] Mike Drob commented on HBASE-18365: --- +1 reviewed master, branch-1, branch-1.3, branch-1.2 > Eliminate the findbugs warnings for hbase-common > > > Key: HBASE-18365 > URL: https://issues.apache.org/jira/browse/HBASE-18365 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18365.branch-1.2.v0.patch, > HBASE-18365.branch-1.3.v0.patch, HBASE-18365.branch-1.v0.patch, > HBASE-18465.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18355) Enable export snapshot tests that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov reassigned HBASE-18355: - Assignee: Vladimir Rodionov > Enable export snapshot tests that were disabled by Proc-V2 AM in HBASE-14614 > > > Key: HBASE-18355 > URL: https://issues.apache.org/jira/browse/HBASE-18355 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > > The Proc-V2 AM in HBASE-14614 disabled the following tests: > - Disabled TestExportSnapshot Hangs. > - Disabled TestSecureExportSnapshot > - Disabled TestMobSecureExportSnapshot and TestMobExportSnapshot > This JIRA tracks the work to enable them. If MOB requires more work, we > could split to 2 tickets. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18357) Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov reassigned HBASE-18357: - Assignee: Vladimir Rodionov > Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in > HBASE-14614 > > > Key: HBASE-18357 > URL: https://issues.apache.org/jira/browse/HBASE-18357 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Vladimir Rodionov > > The Core Proc-V2 AM change in HBASE-14614 disabled two tests inTestHCM: > testMulti and testRegionCaching > This JIRA tracks the work to enable them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18339) Update test-patch to use hadoop 3.0.0-alpha4
[ https://issues.apache.org/jira/browse/HBASE-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18339: -- Attachment: HBASE-18339.patch Update pom and yetus personality. > Update test-patch to use hadoop 3.0.0-alpha4 > > > Key: HBASE-18339 > URL: https://issues.apache.org/jira/browse/HBASE-18339 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18339.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18344) Replace Append#add(byte[], byte[], byte[]) by Append#addColumn
[
https://issues.apache.org/jira/browse/HBASE-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084478#comment-16084478
]
Hadoop QA commented on HBASE-18344:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 23 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
45s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 44s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
46s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}119m 42s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
0s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
56s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
2s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}186m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18344 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876867/HBASE-18344.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux cfe5dc99b3a3 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cb5299a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |
https://builds.apache.o
[jira] [Updated] (HBASE-18260) Address new license dependencies from hadoop3-alpha4
[ https://issues.apache.org/jira/browse/HBASE-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18260: Resolution: Fixed Fix Version/s: 2.0.0-alpha-2 3.0.0 Status: Resolved (was: Patch Available) nope! pushed to branch-2+ > Address new license dependencies from hadoop3-alpha4 > > > Key: HBASE-18260 > URL: https://issues.apache.org/jira/browse/HBASE-18260 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18260.patch > > > hadoop3-alpha4 adds at least one new dependency that requires an update to > our supplemental-models.xml > use this jira to track the changes for after that is released. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18339) Update test-patch to use hadoop 3.0.0-alpha4
[ https://issues.apache.org/jira/browse/HBASE-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084472#comment-16084472 ] Hadoop QA commented on HBASE-18339: --- (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HBASE-Build/7639/console in case of problems. > Update test-patch to use hadoop 3.0.0-alpha4 > > > Key: HBASE-18339 > URL: https://issues.apache.org/jira/browse/HBASE-18339 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18339.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18339) Update test-patch to use hadoop 3.0.0-alpha4
[ https://issues.apache.org/jira/browse/HBASE-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18339: -- Status: Patch Available (was: Open) > Update test-patch to use hadoop 3.0.0-alpha4 > > > Key: HBASE-18339 > URL: https://issues.apache.org/jira/browse/HBASE-18339 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-18339.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18370) Master should attempt reassignment of regions in FAILED_OPEN state
[ https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084424#comment-16084424 ] Andrew Purtell commented on HBASE-18370: This could be an interesting one. If agreeable and doable, we want to be able to do this in branch-1 / branch-1.4 as well as branch-2 and up, so there will be two implementations required. > Master should attempt reassignment of regions in FAILED_OPEN state > -- > > Key: HBASE-18370 > URL: https://issues.apache.org/jira/browse/HBASE-18370 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell > > Currently once a region goes into FAILED_OPEN state this requires operator > intervention. With some underlying causes, this is necessary. With others, > the master could eventually successfully deploy the region without humans in > the loop. The master should optionally attempt automatic resolution of > FAILED_OPEN states with a strategy of: delay, unassign, reassign. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18370) Master should attempt reassignment of regions in FAILED_OPEN state
Andrew Purtell created HBASE-18370: -- Summary: Master should attempt reassignment of regions in FAILED_OPEN state Key: HBASE-18370 URL: https://issues.apache.org/jira/browse/HBASE-18370 Project: HBase Issue Type: Improvement Reporter: Andrew Purtell Currently once a region goes into FAILED_OPEN state this requires operator intervention. With some underlying causes, this is necessary. With others, the master could eventually successfully deploy the region without humans in the loop. The master should optionally attempt automatic resolution of FAILED_OPEN states with a strategy of: delay, unassign, reassign. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18365) Eliminate the findbugs warnings for hbase-common
[ https://issues.apache.org/jira/browse/HBASE-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084343#comment-16084343 ] Chia-Ping Tsai commented on HBASE-18365: Ping for reviews~ > Eliminate the findbugs warnings for hbase-common > > > Key: HBASE-18365 > URL: https://issues.apache.org/jira/browse/HBASE-18365 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18365.branch-1.2.v0.patch, > HBASE-18365.branch-1.3.v0.patch, HBASE-18365.branch-1.v0.patch, > HBASE-18465.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18330) NPE in ReplicationZKLockCleanerChore
[
https://issues.apache.org/jira/browse/HBASE-18330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084406#comment-16084406
]
Andrew Purtell commented on HBASE-18330:
TestRSKilledWhenInitializing failure isn't related, see HBASE-18346
I'm going to commit this trivial fix later today unless objection .
> NPE in ReplicationZKLockCleanerChore
>
>
> Key: HBASE-18330
> URL: https://issues.apache.org/jira/browse/HBASE-18330
> Project: HBase
> Issue Type: Bug
> Components: master, Replication
>Affects Versions: 1.2.5
>Reporter: Minwoo Kang
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18330-branch-1.patch, HBASE-18330-branch-2.patch,
> HBASE-18330-master.patch
>
>
> While I am watching HMaster log, I found NullPointerException Logs.
> This occurs every minute.
>
> 2017-07-06 09:05:02,579 DEBUG [,1498445640728_ChoreService_1]
> cleaner.CleanerChore: Removing: hdfs://*** from archive
> 2017-07-06 09:05:02,585 ERROR [,1498445640728_ChoreService_2]
> hbase.ScheduledChore: Caught error
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.cleaner.ReplicationZKLockCleanerChore.chore(ReplicationZKLockCleanerChore.java:80)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:110)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2017-07-06 09:05:02,585 DEBUG [,1498445640728_ChoreService_1]
> cleaner.CleanerChore: Removing: hdfs://*** from archive
> 2017-07-06 09:05:02,586 DEBUG [,1498445640728_ChoreService_1]
> cleaner.CleanerChore: Removing: hdfs://*** from archive
>
> Here is related code:
> List replicators = queues.getListOfReplicators();
> for (String replicator: replicators) {
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18229) create new Async Split API to embrace AM v2
[ https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18229: - Attachment: hbase-18229-master-v8.patch new patch attached > create new Async Split API to embrace AM v2 > --- > > Key: HBASE-18229 > URL: https://issues.apache.org/jira/browse/HBASE-18229 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0-alpha-2 > > Attachments: HBase-18229-master-v1.patch, > hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch, > hbase-18229-master-v4.patch, hbase-18229-master-v5.patch, > hbase-18229-master-v6.patch, hbase-18229-master-v7.patch, > hbase-18229-master-v7.patch, hbase-18229-master-v8.patch > > > See HBASE-18105 for related information, this jira will change the logic of > Path 1 in splitProcedure, the execution path will be: > *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion -> > MasterRpcServices.splitRegion -> HMaster.splitRegion-> > AssignmentManager.submitProcedure* > Master Will no longer ask Rs and then Rs turn around to ask master to do the > split operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-17818) On branch-1 in module Server, TestRSKilledWhenInitializing and TestScannerHeartbeatMessages broken
[ https://issues.apache.org/jira/browse/HBASE-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-17818. Resolution: Duplicate Fix Version/s: (was: 1.4.0) HBASE-16051 and HBASE-18346 now both dup this. Let's follow up there. > On branch-1 in module Server, TestRSKilledWhenInitializing and > TestScannerHeartbeatMessages broken > -- > > Key: HBASE-17818 > URL: https://issues.apache.org/jira/browse/HBASE-17818 > Project: HBase > Issue Type: Bug > Environment: OS: Ubuntu 14.04 > Arch: x86_64 >Reporter: Anup Halarnkar > > Branch: branch-1 > Command: mvn clean install -X -fn > Output: > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 2.030 s > [INFO] Finished at: 2017-03-22T19:17:16+05:30 > [INFO] Final Memory: 15M/304M > [INFO] > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Apache HBase ... SUCCESS [ 24.315 > s] > [INFO] Apache HBase - Checkstyle .. SUCCESS [ 7.797 > s] > [INFO] Apache HBase - Resource Bundle . SUCCESS [ 0.263 > s] > [INFO] Apache HBase - Annotations . SUCCESS [ 2.416 > s] > [INFO] Apache HBase - Protocol SUCCESS [ 12.994 > s] > [INFO] Apache HBase - Common .. SUCCESS [02:31 > min] > [INFO] Apache HBase - Procedure ... SUCCESS [03:37 > min] > [INFO] Apache HBase - Client .. SUCCESS [01:53 > min] > [INFO] Apache HBase - Hadoop Compatibility SUCCESS [ 8.560 > s] > [INFO] Apache HBase - Hadoop Two Compatibility SUCCESS [ 10.500 > s] > [INFO] Apache HBase - Prefix Tree . SUCCESS [ 9.367 > s] > [INFO] Apache HBase - Server .. FAILURE [ 02:04 > h] > [INFO] Apache HBase - Testing Util SUCCESS [ 5.107 > s] > [INFO] Apache HBase - Thrift .. SUCCESS [04:52 > min] > [INFO] Apache HBase - Rest SUCCESS [13:56 > min] > [INFO] Apache HBase - Shell ... SUCCESS [ 3.980 > s] > [INFO] Apache HBase - Integration Tests ... SUCCESS [ 02:33 > h] > [INFO] Apache HBase - Examples SUCCESS [ 11.129 > s] > [INFO] Apache HBase - External Block Cache SUCCESS [ 1.231 > s] > [INFO] Apache HBase - Assembly FAILURE [ 4.063 > s] > [INFO] Apache HBase - Shaded .. SUCCESS [ 0.172 > s] > [INFO] Apache HBase - Shaded - Client . SUCCESS [ 0.793 > s] > [INFO] Apache HBase - Shaded - Server . SUCCESS [ 1.721 > s] > [INFO] Apache HBase - Archetypes .. SUCCESS [ 0.094 > s] > [INFO] Apache HBase - Exemplar for hbase-client archetype . SUCCESS [01:24 > min] > [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SUCCESS > [01:15 min] > [INFO] Apache HBase - Archetype builder ... SUCCESS [ 31.495 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 05:10 h > [INFO] Finished at: 2017-03-22T19:17:16+05:30 > --- > Failed tests: > TestRSKilledWhenInitializing.testRSTerminationAfterRegisteringToMasterBeforeCreatingEphemeralNode:123 > null > > TestScannerHeartbeatMessages.testScannerHeartbeatMessages:207->testImportanceOfHeartbeats:237 > Heartbeats messages are disabled, an exception should be thrown. If an > exception is not thrown, the test case is not testing the importance of > heartbeat messages > Tests run: 1698, Failures: 2, Errors: 0, Skipped: 11 > -- > If these tests are passing on your end,then please let me know the possible > issue that I could have on my side. > Thanks in advance, > Anup -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18365) Eliminate the findbugs warnings for hbase-common
[
https://issues.apache.org/jira/browse/HBASE-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084342#comment-16084342
]
Hadoop QA commented on HBASE-18365:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 8m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 38s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
45s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:2fc26cc |
| JIRA Issue | HBASE-18365 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876888/HBASE-18365.branch-1.2.v0.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b50086d29322 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patch
[jira] [Updated] (HBASE-18365) Eliminate the findbugs warnings for hbase-common
[ https://issues.apache.org/jira/browse/HBASE-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18365: --- Status: Open (was: Patch Available) > Eliminate the findbugs warnings for hbase-common > > > Key: HBASE-18365 > URL: https://issues.apache.org/jira/browse/HBASE-18365 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18365.branch-1.2.v0.patch, > HBASE-18365.branch-1.3.v0.patch, HBASE-18365.branch-1.v0.patch, > HBASE-18465.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18365) Eliminate the findbugs warnings for hbase-common
[ https://issues.apache.org/jira/browse/HBASE-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18365: --- Attachment: HBASE-18365.branch-1.2.v0.patch > Eliminate the findbugs warnings for hbase-common > > > Key: HBASE-18365 > URL: https://issues.apache.org/jira/browse/HBASE-18365 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18365.branch-1.2.v0.patch, > HBASE-18365.branch-1.3.v0.patch, HBASE-18365.branch-1.v0.patch, > HBASE-18465.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18365) Eliminate the findbugs warnings for hbase-common
[ https://issues.apache.org/jira/browse/HBASE-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18365: --- Status: Patch Available (was: Open) > Eliminate the findbugs warnings for hbase-common > > > Key: HBASE-18365 > URL: https://issues.apache.org/jira/browse/HBASE-18365 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2 > > Attachments: HBASE-18365.branch-1.2.v0.patch, > HBASE-18365.branch-1.3.v0.patch, HBASE-18365.branch-1.v0.patch, > HBASE-18465.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12280) [89-fb] Make the number of blocking store files online configurable
[ https://issues.apache.org/jira/browse/HBASE-12280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084297#comment-16084297 ] Mike Drob commented on HBASE-12280: --- [~youknowjack] - is this still relevant? Can we reframe it in terms of master/branch-2? > [89-fb] Make the number of blocking store files online configurable > --- > > Key: HBASE-12280 > URL: https://issues.apache.org/jira/browse/HBASE-12280 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.89-fb >Reporter: Jack Langman > > This change allows us to change the number of blocking store files on-line. > This is already done, and should appear in the 89-fb branch soon. For > context, see: HBASE-8544, HBASE-8576, HBASE-8805 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084290#comment-16084290
]
Chia-Ping Tsai commented on HBASE-18368:
bq. We shouldn't return NEXT_ROW when FamilyFilter doesn't match.
I copy the explanation from HBASE-13122.
{noformat}
NEXT_ROW can work is because there is regionscanner and store scanner, next_row
affect store scanner.
When one storescanner switch to next row, the region scanner switch to the next
storescanner, the second store scanner will continue check the current row.
That why the change can be more efficiency.
{noformat}
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reassigned HBASE-18368:
--
Assignee: Allan Yang
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Assignee: Allan Yang
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084261#comment-16084261
]
Chia-Ping Tsai commented on HBASE-18368:
bq. this bug may not be introduced by HBASE-17678, but by HBASE-13122.
You are right.
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18368) FamilyFilters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18368:
---
Summary: FamilyFilters with OR does not work (was: Filters with OR does
not work)
> FamilyFilters with OR does not work
> ---
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084253#comment-16084253
]
Ted Yu commented on HBASE-18368:
Looks like the patch was generated on Windows.
Please use dos2unix or some similar tool to remove the trailing Ctrl-M.
> Filters with OR does not work
> -
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18368) Filters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-18368:
---
Attachment: HBASE-18368.branch-1.patch
> Filters with OR does not work
> -
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084242#comment-16084242
]
Allan Yang commented on HBASE-18368:
Did some research, this bug may not be introduced by HBASE-17678, but by
HBASE-13122. Upload a patch to fix this. We shouldn't return NEXT_ROW when
FamilyFilter doesn't match.
> Filters with OR does not work
> -
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
> Attachments: HBASE-18368.branch-1.patch
>
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-13287) Append is missing methods, such as addColumn()
[
https://issues.apache.org/jira/browse/HBASE-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-13287.
Resolution: Duplicate
> Append is missing methods, such as addColumn()
> --
>
> Key: HBASE-13287
> URL: https://issues.apache.org/jira/browse/HBASE-13287
> Project: HBase
> Issue Type: Bug
> Components: API
>Affects Versions: 1.0.0
>Reporter: Lars George
>Assignee: Jan Hentschel
>
> Analog to Put, Delete, Get, etc. Append should have the same common method
> signature, namely {{addColumn()}} as opposed to the older {{add()}}. Also
> consider adding the other variant that Put has, including the _immutable_
> versions.
> Question related: should those methods not better be in the Mutation class?
> All derived classes implement them. {{setTTL()}} is in Mutation, and throws
> an exception when used with {{Delete}}. So why not moved those down too? (cc
> [~saint@gmail.com])
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18344) Replace Append#add(byte[], byte[], byte[]) by Append#addColumn
[ https://issues.apache.org/jira/browse/HBASE-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084208#comment-16084208 ] Chia-Ping Tsai commented on HBASE-18344: +1 for v2 > Replace Append#add(byte[], byte[], byte[]) by Append#addColumn > -- > > Key: HBASE-18344 > URL: https://issues.apache.org/jira/browse/HBASE-18344 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Jan Hentschel >Priority: Trivial > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18344.master.001.patch, > HBASE-18344.master.002.patch > > > We have Put#addColumn and Increment#addColumn but there is no > Append#addColumn. We should add Append#addColumn for consistency. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18061) [C++] Fix retry logic in multi-get calls
[ https://issues.apache.org/jira/browse/HBASE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084209#comment-16084209 ] Ted Yu commented on HBASE-18061: I tried to apply patch v7 but saw: 1 out of 16 hunks FAILED -- saving rejects to file hbase-native-client/core/async-batch-rpc-retrying-caller.cc.rej > [C++] Fix retry logic in multi-get calls > > > Key: HBASE-18061 > URL: https://issues.apache.org/jira/browse/HBASE-18061 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Sudeep Sunthankar > Fix For: HBASE-14850 > > Attachments: HBASE-18061.HBASE-14850.v1.patch, > HBASE-18061.HBASE-14850.v3.patch, HBASE-18061.HBASE-14850.v5.patch, > HBASE-18061.HBASE-14850.v6.patch, HBASE-18061.HBASE-14850.v7.patch > > > HBASE-17576 adds multi-gets. There are a couple of todos to fix in the retry > logic, and some unit testing to be done for the multi-gets. We'll do these in > this issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13287) Append is missing methods, such as addColumn()
[
https://issues.apache.org/jira/browse/HBASE-13287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084206#comment-16084206
]
Chia-Ping Tsai commented on HBASE-13287:
bq. So why not moved those down too?
The sub class implements the setTTL() for overriding the return type.
Will mark this as duplicate and than close this.
> Append is missing methods, such as addColumn()
> --
>
> Key: HBASE-13287
> URL: https://issues.apache.org/jira/browse/HBASE-13287
> Project: HBase
> Issue Type: Bug
> Components: API
>Affects Versions: 1.0.0
>Reporter: Lars George
>Assignee: Jan Hentschel
>
> Analog to Put, Delete, Get, etc. Append should have the same common method
> signature, namely {{addColumn()}} as opposed to the older {{add()}}. Also
> consider adding the other variant that Put has, including the _immutable_
> versions.
> Question related: should those methods not better be in the Mutation class?
> All derived classes implement them. {{setTTL()}} is in Mutation, and throws
> an exception when used with {{Delete}}. So why not moved those down too? (cc
> [~saint@gmail.com])
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084167#comment-16084167
]
Allan Yang commented on HBASE-18368:
Sorry, [~chia7712], you are right, HBASE-17678 is the root cause. I haven't
update my local branch, my previous comment is wrong.
> Filters with OR does not work
> -
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18364) Downgrade surefire
[ https://issues.apache.org/jira/browse/HBASE-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18364: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks folks :) Pushed to branch-2 and master. > Downgrade surefire > -- > > Key: HBASE-18364 > URL: https://issues.apache.org/jira/browse/HBASE-18364 > Project: HBase > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-18364.001.patch > > > Seems like 2.20 broke package-wildcard for test specification. Pull this back > to something that works. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18358) Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish' to branch-1.3
[
https://issues.apache.org/jira/browse/HBASE-18358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084171#comment-16084171
]
Ted Yu commented on HBASE-18358:
[~chia7712]:
Thanks for the review, too.
> Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent
> Region#flush() to finish' to branch-1.3
> --
>
> Key: HBASE-18358
> URL: https://issues.apache.org/jira/browse/HBASE-18358
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: 18358.branch-1.3.patch, 18358.v2.txt, 18358.v3.txt
>
>
> HBASE-18099 was only integrated to branch-1 and above in consideration of
> backward compatibility.
> This issue is to backport the fix to branch-1.3 and branch-1.2.
> Quoting Gary's suggestion from the tail of HBASE-18099 :
> {quote}
> Sure, don't add the method to Region, just to HRegion, check for an instance
> of HRegion in FlushSnapshotSubprocedure and cast the instance before calling
> the method.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18344) Replace Append#add(byte[], byte[], byte[]) by Append#addColumn
[ https://issues.apache.org/jira/browse/HBASE-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Hentschel updated HBASE-18344: -- Attachment: HBASE-18344.master.002.patch > Replace Append#add(byte[], byte[], byte[]) by Append#addColumn > -- > > Key: HBASE-18344 > URL: https://issues.apache.org/jira/browse/HBASE-18344 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Jan Hentschel >Priority: Trivial > Labels: beginner > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18344.master.001.patch, > HBASE-18344.master.002.patch > > > We have Put#addColumn and Increment#addColumn but there is no > Append#addColumn. We should add Append#addColumn for consistency. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17678) FilterList with MUST_PASS_ONE may lead to redundant cells returned
[
https://issues.apache.org/jira/browse/HBASE-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084125#comment-16084125
]
Chia-Ping Tsai commented on HBASE-17678:
A bug may be introduced by this jira. see HBASE-18368
> FilterList with MUST_PASS_ONE may lead to redundant cells returned
> --
>
> Key: HBASE-17678
> URL: https://issues.apache.org/jira/browse/HBASE-17678
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 1.3.0, 1.2.1
> Environment: RedHat 7.x
>Reporter: Jason Tokayer
>Assignee: Zheng Hu
> Attachments: HBASE-17678.addendum.patch, HBASE-17678.addendum.patch,
> HBASE-17678.branch-1.1.v1.patch, HBASE-17678.branch-1.1.v2.patch,
> HBASE-17678.branch-1.1.v2.patch, HBASE-17678.branch-1.1.v2.patch,
> HBASE-17678.branch-1.v1.patch, HBASE-17678.branch-1.v1.patch,
> HBASE-17678.branch-1.v2.patch, HBASE-17678.branch-1.v2.patch,
> HBASE-17678.v1.patch, HBASE-17678.v1.rough.patch, HBASE-17678.v2.patch,
> HBASE-17678.v3.patch, HBASE-17678.v4.patch, HBASE-17678.v4.patch,
> HBASE-17678.v5.patch, HBASE-17678.v6.patch, HBASE-17678.v7.patch,
> HBASE-17678.v7.patch, TestColumnPaginationFilterDemo.java
>
>
> When combining ColumnPaginationFilter with a single-element filterList,
> MUST_PASS_ONE and MUST_PASS_ALL give different results when there are
> multiple cells with the same timestamp. This is unexpected since there is
> only a single filter in the list, and I would believe that MUST_PASS_ALL and
> MUST_PASS_ONE should only affect the behavior of the joined filter and not
> the behavior of any one of the individual filters. If this is not a bug then
> it would be nice if the documentation is updated to explain this nuanced
> behavior.
> I know that there was a decision made in an earlier Hbase version to keep
> multiple cells with the same timestamp. This is generally fine but presents
> an issue when using the aforementioned filter combination.
> Steps to reproduce:
> In the shell create a table and insert some data:
> {code:none}
> create 'ns:tbl',{NAME => 'family',VERSIONS => 100}
> put 'ns:tbl','row','family:name','John',1
> put 'ns:tbl','row','family:name','Jane',1
> put 'ns:tbl','row','family:name','Gil',1
> put 'ns:tbl','row','family:name','Jane',1
> {code}
> Then, use a Scala client as:
> {code:none}
> import org.apache.hadoop.hbase.filter._
> import org.apache.hadoop.hbase.util.Bytes
> import org.apache.hadoop.hbase.client._
> import org.apache.hadoop.hbase.{CellUtil, HBaseConfiguration, TableName}
> import scala.collection.mutable._
> val config = HBaseConfiguration.create()
> config.set("hbase.zookeeper.quorum", "localhost")
> config.set("hbase.zookeeper.property.clientPort", "2181")
> val connection = ConnectionFactory.createConnection(config)
> val logicalOp = FilterList.Operator.MUST_PASS_ONE
> val limit = 1
> var resultsList = ListBuffer[String]()
> for (offset <- 0 to 20 by limit) {
> val table = connection.getTable(TableName.valueOf("ns:tbl"))
> val paginationFilter = new ColumnPaginationFilter(limit,offset)
> val filterList: FilterList = new FilterList(logicalOp,paginationFilter)
> println("@ filterList = "+filterList)
> val results = table.get(new
> Get(Bytes.toBytes("row")).setFilter(filterList))
> val cells = results.rawCells()
> if (cells != null) {
> for (cell <- cells) {
> val value = new String(CellUtil.cloneValue(cell))
> val qualifier = new String(CellUtil.cloneQualifier(cell))
> val family = new String(CellUtil.cloneFamily(cell))
> val result = "OFFSET = "+offset+":"+family + "," + qualifier
> + "," + value + "," + cell.getTimestamp()
> resultsList.append(result)
> }
> }
> }
> resultsList.foreach(println)
> {code}
> Here are the results for different limit and logicalOp settings:
> {code:none}
> Limit = 1 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 1 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 1:family,name,Gil,1
> OFFSET = 2:family,name,Jane,1
> OFFSET = 3:family,name,John,1
> Limit = 2 & logicalOp = MUST_PASS_ALL:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> Limit = 2 & logicalOp = MUST_PASS_ONE:
> scala> resultsList.foreach(println)
> OFFSET = 0:family,name,Jane,1
> OFFSET = 2:family,name,Jane,1
> {code}
> So, it seems that MUST_PASS_ALL gives the expected behavior, but
> MUST_PASS_ONE does not. Furthermore, MUST_PASS_O
[jira] [Commented] (HBASE-18368) Filters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084129#comment-16084129
]
Allan Yang commented on HBASE-18368:
Which HBase version are you using, I didn't reproduce your bug in both master
branch and branch-1 using your code.
> Filters with OR does not work
> -
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18368) Filters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18368:
---
Affects Version/s: (was: 2.0.0)
2.0.0-alpha-1
> Filters with OR does not work
> -
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Peter Somogyi
>Priority: Critical
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18368) Filters with OR does not work
[
https://issues.apache.org/jira/browse/HBASE-18368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084124#comment-16084124
]
Chia-Ping Tsai commented on HBASE-18368:
If the HBASE-17678 is reverted, the UT will work well.
ping [~openinx]. Would you please take a look?
> Filters with OR does not work
> -
>
> Key: HBASE-18368
> URL: https://issues.apache.org/jira/browse/HBASE-18368
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Peter Somogyi
>Priority: Critical
>
> Scan gives back incomplete list if multiple filters are combined with OR /
> MUST_PASS_ONE.
> Using 2 FamilyFilters in a FilterList using MUST_PASS_ONE operator will give
> back results for only the first Filter.
> {code:java|title=Test code}
> @Test
> public void testFiltersWithOr() throws Exception {
> TableName tn = TableName.valueOf("MyTest");
> Table table = utility.createTable(tn, new String[] {"cf1", "cf2"});
> byte[] CF1 = Bytes.toBytes("cf1");
> byte[] CF2 = Bytes.toBytes("cf2");
> Put put1 = new Put(Bytes.toBytes("0"));
> put1.addColumn(CF1, Bytes.toBytes("col_a"), Bytes.toBytes(0));
> table.put(put1);
> Put put2 = new Put(Bytes.toBytes("0"));
> put2.addColumn(CF2, Bytes.toBytes("col_b"), Bytes.toBytes(0));
> table.put(put2);
> FamilyFilter filterCF1 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF1));
> FamilyFilter filterCF2 = new FamilyFilter(CompareFilter.CompareOp.EQUAL,
> new BinaryComparator(CF2));
> FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ONE);
> filterList.addFilter(filterCF1);
> filterList.addFilter(filterCF2);
> Scan scan = new Scan();
> scan.setFilter(filterList);
> ResultScanner scanner = table.getScanner(scan);
> System.out.println(filterList);
> for (Result rr = scanner.next(); rr != null; rr = scanner.next()) {
> System.out.println(rr);
> }
> }
> {code}
> {noformat:title=Output}
> FilterList OR (2/2): [FamilyFilter (EQUAL, cf1), FamilyFilter (EQUAL, cf2)]
> keyvalues={0/cf1:col_a/1499852754957/Put/vlen=4/seqid=0}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18364) Downgrade surefire
[ https://issues.apache.org/jira/browse/HBASE-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084123#comment-16084123 ] Josh Elser commented on HBASE-18364: Thanks folks. Let me push this -- we can continue to monitor. > Downgrade surefire > -- > > Key: HBASE-18364 > URL: https://issues.apache.org/jira/browse/HBASE-18364 > Project: HBase > Issue Type: Task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-18364.001.patch > > > Seems like 2.20 broke package-wildcard for test specification. Pull this back > to something that works. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17442: --- Attachment: HBASE-17442.v1.patch > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0 > > Attachments: 0001-hbase-replication-module.patch, HBASE-17442.v1.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[ https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083988#comment-16083988 ] Guanghao Zhang commented on HBASE-17442: [~stack] I attach a v1 patch which only move the replication related classes to hbase-server package. > Move most of the replication related classes to hbase-server package > > > Key: HBASE-17442 > URL: https://issues.apache.org/jira/browse/HBASE-17442 > Project: HBase > Issue Type: Sub-task > Components: build, Replication >Affects Versions: 2.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0 > > Attachments: 0001-hbase-replication-module.patch, HBASE-17442.v1.patch > > > After the replication requests are routed through master, replication > implementation details didn't need be exposed to client. We should move most > of the replication related classes to hbase-server package. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17442) Move most of the replication related classes to hbase-server package
[
https://issues.apache.org/jira/browse/HBASE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083940#comment-16083940
]
Hadoop QA commented on HBASE-17442:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
35s{color} | {color:red} hbase-server in master has 9 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 34s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 17s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestServerCrashProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-17442 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12876800/HBASE-17442.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e6bc54558db8 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cb5299a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/7635/artifact/patchprocess/branch-find
[jira] [Commented] (HBASE-18177) FanOutOneBlockAsyncDFSOutputHelper fails to compile against Hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083900#comment-16083900 ] Hudson commented on HBASE-18177: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3361 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3361/]) HBASE-18177 FanOutOneBlockAsyncDFSOutputHelper fails to compile against (zhangduo: rev cb5299ae9b3360a6cca93958a74417d663135a60) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java > FanOutOneBlockAsyncDFSOutputHelper fails to compile against Hadoop 3 > > > Key: HBASE-18177 > URL: https://issues.apache.org/jira/browse/HBASE-18177 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Esteban Gutierrez >Assignee: Mike Drob > Fix For: 2.0.0-alpha-2 > > Attachments: HBASE-18177.patch, HBASE-18177.v2.patch > > > After HDFS-10996 ClientProtocol#create() needs to specify the erasure code > policy to use. In the meantime we should add a workaround to > FanOutOneBlockAsyncDFSOutputHelper to be able to compile against Hadoop 3 and > Hadoop 2. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17738) BucketCache startup is slow
[ https://issues.apache.org/jira/browse/HBASE-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083877#comment-16083877 ] ramkrishna.s.vasudevan commented on HBASE-17738: bq.The math in calculating the number of BBs per thread seems some thing wrong. I agree. Thanks for the catch. The change was done in my other box and was not put up in my other place. Good catch. bq.ets make the Callable return a BB[] so that u can avoid the start and end index math and changing a shared variable from diff threads. Ok. I will take up ur suggestion and the patch that you gave me here. > BucketCache startup is slow > --- > > Key: HBASE-17738 > URL: https://issues.apache.org/jira/browse/HBASE-17738 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Affects Versions: 2.0.0 >Reporter: stack >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17738_2.patch, HBASE-17738_2.patch, > HBASE-17738_3.patch, HBASE-17738_4.patch, HBASE-17738_5_withoutUnsafe.patch, > HBASE-17738_6_withoutUnsafe.patch, HBASE-17738.patch > > > If you set bucketcache size at 64G say and then start hbase, it takes a long > time. Can we do the allocations in parallel and not inline with the server > startup? > Related, prefetching on a bucketcache is slow. Speed it up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
