[jira] [Commented] (HBASE-18160) Fix incorrect logic in FilterList.filterKeyValue
[
https://issues.apache.org/jira/browse/HBASE-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174350#comment-16174350
]
Hadoop QA commented on HBASE-18160:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
29s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}155m 11s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}217m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18160 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888194/HBASE-18160.v7.patch |
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f3d8d3559cb9 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174335#comment-16174335 ] Chia-Ping Tsai commented on HBASE-18010: It seems the failed tests are related to the patch. [~anastas] WDYT? > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 3.0.0 > > Attachments: HBASE-18010-branch-2.patch, HBASE-18010-V04.patch, > HBASE-18010-V06.patch, HBASE-18010-V07.patch, HBASE-18010-V08.patch, > HBASE-18010-V09.patch, HBASE-18010-V10.patch, HBASE-18010-V11.patch > > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18823) Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
[ https://issues.apache.org/jira/browse/HBASE-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174328#comment-16174328 ] Hudson commented on HBASE-18823: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3751 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3751/]) HBASE-18823 Apply RegionInfo to (appy: rev a6c3c645fd193f1805d449d91528e2a9246f8ff8) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithRemove.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/BaseTestHBaseFsck.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverForAddingMutationsFromCoprocessors.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorMetrics.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ExampleMasterObserverWithMetrics.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/WALObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SampleRegionWALObserver.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/CoprocessorWhitelistMasterObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestEnableTable.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java > Apply RegionInfo to MasterObserver/RegionObserver/WALObserver > - > > Key: HBASE-18823 > URL: https://issues.apache.org/jira/browse/HBASE-18823 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Labels: incompatible > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18823.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17782) Extend IdReadWriteLock to support using both weak and soft reference
[
https://issues.apache.org/jira/browse/HBASE-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174325#comment-16174325
]
Yu Li commented on HBASE-17782:
---
[~chenyechao] I guess you changed the description by accident and have revert
it, please let me know if any concern here. Thanks.
> Extend IdReadWriteLock to support using both weak and soft reference
>
>
> Key: HBASE-17782
> URL: https://issues.apache.org/jira/browse/HBASE-17782
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17782.patch, HBASE-17782.patch
>
>
> As per discussed in HBASE-17747, we need to make it configurable to decide
> whether to use weak or soft reference for {{IdReadWriteLock}}, with soft
> reference by default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17782) Extend IdReadWriteLock to support using both weak and soft reference
[
https://issues.apache.org/jira/browse/HBASE-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-17782:
--
Description: As per discussed in HBASE-17747, we need to make it
configurable to decide whether to use weak or soft reference for
{{IdReadWriteLock}}, with soft reference by default. (was: # As per discussed
in HBASE-17747, we need to make it configurable to decide whether to use weak
or soft reference for {{IdReadWriteLock}}, with soft reference by default.)
> Extend IdReadWriteLock to support using both weak and soft reference
>
>
> Key: HBASE-17782
> URL: https://issues.apache.org/jira/browse/HBASE-17782
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17782.patch, HBASE-17782.patch
>
>
> As per discussed in HBASE-17747, we need to make it configurable to decide
> whether to use weak or soft reference for {{IdReadWriteLock}}, with soft
> reference by default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18807) Remove PB references from Observers for Quotas
[
https://issues.apache.org/jira/browse/HBASE-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174295#comment-16174295
]
Hadoop QA commented on HBASE-18807:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
59s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
15s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 1s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m
21s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:bd219c0 |
| JIRA Issue | HBASE-18807 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888191/HBASE-18807.002.branch-2.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fece260593ac 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/compon
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13346:
--
Attachment: HBASE-13346.master.003.patch
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174284#comment-16174284 ] Hudson commented on HBASE-16478: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3750 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3750/]) Revert "HBASE-16478 Rename WALKey in PB to WALEdit This is a rebase of (stack: rev 37696fffe95bd1cf3c4ca92a4a11560d53df769b) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogReader.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSink.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALKey.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto * (edit) hbase-protocol-shaded/src/main/protobuf/WAL.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174282#comment-16174282
]
stack commented on HBASE-13346:
---
Some feedback on the patch [~tamaas]
You have filterCell calling filterKeyValue in FilterBase. Do you mean to do it
other way around since filterKV is Deprecated? Ditto in FilterWrapper?
SkipFilter does the right thing.
Yeah, might be too much deprecating and renaming filterKv in Import but it
should use the new methods? It does this:
Filter.ReturnCode code = filter.filterKeyValue(c);
Should be filterCell?
No deprecation in MobReferenceOnlyFilter becuase it is new to 2.0?
AccessControlFilter? Is that new in 2.0? and VisibilityController? Or maybe
these are private classes?
You don't want to move off the deprecated methods in Tests?
-kv = new KeyValue(ROW, COLUMN_FAMILY, COLUMN_QUALIFIER_3, VAL_1);
-System.out.println(filter.filterKeyValue(kv));
+cell = new KeyValue(ROW, COLUMN_FAMILY, COLUMN_QUALIFIER_3, VAL_1);
+System.out.println(filter.filterKeyValue(cell));
There are a bunch. This is a nit in scheme of things.
Otherwise, patch is great. Let me retry it.
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-14914) Move functionality needed to handle Snapshots without RegionServer dependency
[ https://issues.apache.org/jira/browse/HBASE-14914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14914: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-11843) > Move functionality needed to handle Snapshots without RegionServer dependency > - > > Key: HBASE-14914 > URL: https://issues.apache.org/jira/browse/HBASE-14914 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Nate Edel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-11843) MapReduce classes shouldn't be in hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-11843. --- Resolution: Duplicate Resolving as duplicate of HBASE-18640 "Move mapreduce out of hbase-server into separate module" though HBASE-18640 came later. This issue had great subtasks that I moved out as standalone. They are linked here. > MapReduce classes shouldn't be in hbase-server > -- > > Key: HBASE-11843 > URL: https://issues.apache.org/jira/browse/HBASE-11843 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Keegan Witt >Assignee: Nate Edel >Priority: Critical > Fix For: 2.0.0 > > > I'm opening this to continue the discussion I started a while back: > http://mail-archives.apache.org/mod_mbox/hbase-user/201405.mbox/%3ccamuu0w_xooxeg779rrfjturau+uxeavunzxkw9dxfo-gh5y...@mail.gmail.com%3E. > To summarize, I think the MapReduce classes used by clients (like > TableMapper, TableReducer, etc) don't belong in hbase-server. This forces > the user to pull in a rather large artifact for a relatively small number of > classes. These should either be put in hbase-client, or possibly an artifact > of their own (like the hbase-mapreduce idea Enis suggested). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14913) Enable testing I/O without regionserver or threads.
[ https://issues.apache.org/jira/browse/HBASE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174272#comment-16174272 ] stack commented on HBASE-14913: --- This used to be a subtask but now is standalone > Enable testing I/O without regionserver or threads. > --- > > Key: HBASE-14913 > URL: https://issues.apache.org/jira/browse/HBASE-14913 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Nate Edel >Assignee: Nate Edel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14911) Move functionality needed to write HFiles without RegionServer dependency
[ https://issues.apache.org/jira/browse/HBASE-14911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174270#comment-16174270 ] stack commented on HBASE-14911: --- This used to be a subtask but now is standalone > Move functionality needed to write HFiles without RegionServer dependency > - > > Key: HBASE-14911 > URL: https://issues.apache.org/jira/browse/HBASE-14911 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Nate Edel >Assignee: Nate Edel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14912) Move functionality needed to read WAL without RegionServer dependency
[ https://issues.apache.org/jira/browse/HBASE-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174271#comment-16174271 ] stack commented on HBASE-14912: --- This used to be a subtask but now is standalone > Move functionality needed to read WAL without RegionServer dependency > - > > Key: HBASE-14912 > URL: https://issues.apache.org/jira/browse/HBASE-14912 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Nate Edel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-14913) Enable testing I/O without regionserver or threads.
[ https://issues.apache.org/jira/browse/HBASE-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14913: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-11843) > Enable testing I/O without regionserver or threads. > --- > > Key: HBASE-14913 > URL: https://issues.apache.org/jira/browse/HBASE-14913 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Nate Edel >Assignee: Nate Edel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-14912) Move functionality needed to read WAL without RegionServer dependency
[ https://issues.apache.org/jira/browse/HBASE-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14912: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-11843) > Move functionality needed to read WAL without RegionServer dependency > - > > Key: HBASE-14912 > URL: https://issues.apache.org/jira/browse/HBASE-14912 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Nate Edel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-14911) Move functionality needed to write HFiles without RegionServer dependency
[ https://issues.apache.org/jira/browse/HBASE-14911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14911: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-11843) > Move functionality needed to write HFiles without RegionServer dependency > - > > Key: HBASE-14911 > URL: https://issues.apache.org/jira/browse/HBASE-14911 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Nate Edel >Assignee: Nate Edel > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper
[ https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-11462. --- Resolution: Fixed Re-resolving. This was applied to master/branch-2 and got reopened for a backport to branch-1 that never happened. > MetaTableAccessor shouldn't use ZooKeeeper > -- > > Key: HBASE-11462 > URL: https://issues.apache.org/jira/browse/HBASE-11462 > Project: HBase > Issue Type: Improvement > Components: Client, Zookeeper >Affects Versions: 2.0.0 >Reporter: Mikhail Antonov >Assignee: ryan rawson > Fix For: 2.0.0 > > Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, > HBASE-11462.v4.patch > > > After committing patch for HBASE-4495, there's an further improvement which > can be made (discussed originally on review board to that jira). > We have MetaTableAccessor and MetaTableLocator classes. First one is used to > access information stored in hbase:meta table. Second one is used to deal > with ZooKeeper state to find out region server hosting hbase:meta, wait for > it to become available and so on. > MetaTableAccessor, in turn, should only operate on the meta table content, so > shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when > callers request assignment information, they can request location of meta > table itself, which we can't read from meta, so in that case > MetaTableAccessor relays the call to MetaTableLocator. May be the solution > here is to declare that clients of MetaTableAccessor shall not use it to work > with meta table itself (not it's content). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-11466) HConnectionImplementation should not use ZK
[ https://issues.apache.org/jira/browse/HBASE-11466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174257#comment-16174257 ] stack commented on HBASE-11466: --- Kicking this out. We should do it but it is not being worked on for 2.0. > HConnectionImplementation should not use ZK > --- > > Key: HBASE-11466 > URL: https://issues.apache.org/jira/browse/HBASE-11466 > Project: HBase > Issue Type: Sub-task > Components: Client, Consensus >Affects Versions: 2.0.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Critical > > Currently ConnectionManager.HConnectionImplementation uses ZK to get address > of current master. Should instead use pluggable interface to get location of > master to connect to (current active master in the cluster, until we have > multiple active masters) elsewhere (e.g. round-robin over the list of masters > set in the client's hbase-site.xml). > Currently it uses MasterAddressTracker, which reads from ZK, and this is the > only place where MasterAddressTracker is used on the client side (except > ZkUtil util method which dumps ZK namespace to log). So implementation of > failover proxy which fails over multiple masters will probably used only here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-11462) MetaTableAccessor shouldn't use ZooKeeeper
[ https://issues.apache.org/jira/browse/HBASE-11462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-11462: - Assignee: Mikhail Antonov (was: ryan rawson) > MetaTableAccessor shouldn't use ZooKeeeper > -- > > Key: HBASE-11462 > URL: https://issues.apache.org/jira/browse/HBASE-11462 > Project: HBase > Issue Type: Improvement > Components: Client, Zookeeper >Affects Versions: 2.0.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Fix For: 2.0.0 > > Attachments: HBASE-11462.v4.patch, HBASE-11462.v4.patch, > HBASE-11462.v4.patch > > > After committing patch for HBASE-4495, there's an further improvement which > can be made (discussed originally on review board to that jira). > We have MetaTableAccessor and MetaTableLocator classes. First one is used to > access information stored in hbase:meta table. Second one is used to deal > with ZooKeeper state to find out region server hosting hbase:meta, wait for > it to become available and so on. > MetaTableAccessor, in turn, should only operate on the meta table content, so > shouldn't need ZK. The only reason why MetaTableAccessor is using ZK - when > callers request assignment information, they can request location of meta > table itself, which we can't read from meta, so in that case > MetaTableAccessor relays the call to MetaTableLocator. May be the solution > here is to declare that clients of MetaTableAccessor shall not use it to work > with meta table itself (not it's content). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[
https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174260#comment-16174260
]
Hadoop QA commented on HBASE-18651:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
17s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
40m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18651 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888200/HBASE-18651.master.006.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 84736d3b0416 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / a6c3c64 |
| Default Java | 1.8.0_144 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8721/testReport/ |
| modules | C: hbase-it U: hbase-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8721/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Let ChaosMonkeyRunner expose the chaos monkey runner it creates
>
[jira] [Updated] (HBASE-11466) HConnectionImplementation should not use ZK
[ https://issues.apache.org/jira/browse/HBASE-11466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11466: -- Fix Version/s: (was: 2.0.0) > HConnectionImplementation should not use ZK > --- > > Key: HBASE-11466 > URL: https://issues.apache.org/jira/browse/HBASE-11466 > Project: HBase > Issue Type: Sub-task > Components: Client, Consensus >Affects Versions: 2.0.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Critical > > Currently ConnectionManager.HConnectionImplementation uses ZK to get address > of current master. Should instead use pluggable interface to get location of > master to connect to (current active master in the cluster, until we have > multiple active masters) elsewhere (e.g. round-robin over the list of masters > set in the client's hbase-site.xml). > Currently it uses MasterAddressTracker, which reads from ZK, and this is the > only place where MasterAddressTracker is used on the client side (except > ZkUtil util method which dumps ZK namespace to log). So implementation of > failover proxy which fails over multiple masters will probably used only here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-11468) MasterAddressTracker needs to be moved to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174256#comment-16174256 ] stack commented on HBASE-11468: --- We should do this but it is part of a bigger undo zk from client project that is not being worked on. > MasterAddressTracker needs to be moved to hbase-server > -- > > Key: HBASE-11468 > URL: https://issues.apache.org/jira/browse/HBASE-11468 > Project: HBase > Issue Type: Sub-task > Components: Client, Consensus, Zookeeper >Reporter: Mikhail Antonov >Assignee: Sergey Soldatov > > Later we should get rid of it. For now, for the purpose to abstract client > from ZK, we don't use in in hbase-client (where we read master location > elsewhere), but on the server side we can use it for now, and still keep > current znode tracking location of current active master. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-11468) MasterAddressTracker needs to be moved to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11468: -- Fix Version/s: (was: 2.0.0) > MasterAddressTracker needs to be moved to hbase-server > -- > > Key: HBASE-11468 > URL: https://issues.apache.org/jira/browse/HBASE-11468 > Project: HBase > Issue Type: Sub-task > Components: Client, Consensus, Zookeeper >Reporter: Mikhail Antonov >Assignee: Sergey Soldatov > > Later we should get rid of it. For now, for the purpose to abstract client > from ZK, we don't use in in hbase-client (where we read master location > elsewhere), but on the server side we can use it for now, and still keep > current znode tracking location of current active master. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-11469) MetaTableLocator should be moved to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174255#comment-16174255 ] stack commented on HBASE-11469: --- Unscheduled. We should do this. It wouldn't be hard even but it is part of the bigger project of undoing client from zk. > MetaTableLocator should be moved to hbase-server > > > Key: HBASE-11469 > URL: https://issues.apache.org/jira/browse/HBASE-11469 > Project: HBase > Issue Type: Sub-task > Components: Client, Consensus, Zookeeper >Reporter: Mikhail Antonov >Assignee: Alex Newman > > It shall not be used on client side, clients shall make rpc calls to master > to find out about meta. Also it needs to either be removed from Server > interface available on client side and moved into RegionServer class or > something, or whole Server interface needs to be moved to hbase-server. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-11469) MetaTableLocator should be moved to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11469: -- Fix Version/s: (was: 2.0.0) > MetaTableLocator should be moved to hbase-server > > > Key: HBASE-11469 > URL: https://issues.apache.org/jira/browse/HBASE-11469 > Project: HBase > Issue Type: Sub-task > Components: Client, Consensus, Zookeeper >Reporter: Mikhail Antonov >Assignee: Alex Newman > > It shall not be used on client side, clients shall make rpc calls to master > to find out about meta. Also it needs to either be removed from Server > interface available on client side and moved into RegionServer class or > something, or whole Server interface needs to be moved to hbase-server. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17782) Extend IdReadWriteLock to support using both weak and soft reference
[
https://issues.apache.org/jira/browse/HBASE-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yechao Chen updated HBASE-17782:
Description: # As per discussed in HBASE-17747, we need to make it
configurable to decide whether to use weak or soft reference for
{{IdReadWriteLock}}, with soft reference by default. (was: As per discussed in
HBASE-17747, we need to make it configurable to decide whether to use weak or
soft reference for {{IdReadWriteLock}}, with soft reference by default.)
> Extend IdReadWriteLock to support using both weak and soft reference
>
>
> Key: HBASE-17782
> URL: https://issues.apache.org/jira/browse/HBASE-17782
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17782.patch, HBASE-17782.patch
>
>
> # As per discussed in HBASE-17747, we need to make it configurable to decide
> whether to use weak or soft reference for {{IdReadWriteLock}}, with soft
> reference by default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18806) VerifyRep by snapshot need not to restore snapshot for each mapper
[
https://issues.apache.org/jira/browse/HBASE-18806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174251#comment-16174251
]
Zheng Hu commented on HBASE-18806:
--
Upload patch v3 for fixing the v2's comment in RB.
bq. Have you tried it (or backport) on cluster ?
Not backport to our cluster yet. Let's see the result of Hadoop QA first.
Later I'll try to test in our cluster.
> VerifyRep by snapshot need not to restore snapshot for each mapper
> --
>
> Key: HBASE-18806
> URL: https://issues.apache.org/jira/browse/HBASE-18806
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0-alpha-2
>Reporter: Zheng Hu
>Assignee: Zheng Hu
> Attachments: HBASE-18806.v1.patch, HBASE-18806.v2.patch,
> HBASE-18806.v3.patch
>
>
> In following method stack, seems like each mapper task will restore the
> snapshot. If we verify replication by a snapshot which has many hfiles,
> then we will take long time to restore snapshot. In our cluster, we took
> ~30min for the snapshot restoring when verify a big table.
> {code}
> Verifier.map
> |> replicatedScanner = new TableSnapshotScanner(...)
> |>
> TableSnapshotScanner.init()
>
> |-> RestoreSnapshotHelper.copySnapshotForScanner
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18806) VerifyRep by snapshot need not to restore snapshot for each mapper
[
https://issues.apache.org/jira/browse/HBASE-18806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-18806:
-
Attachment: HBASE-18806.v3.patch
> VerifyRep by snapshot need not to restore snapshot for each mapper
> --
>
> Key: HBASE-18806
> URL: https://issues.apache.org/jira/browse/HBASE-18806
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0-alpha-2
>Reporter: Zheng Hu
>Assignee: Zheng Hu
> Attachments: HBASE-18806.v1.patch, HBASE-18806.v2.patch,
> HBASE-18806.v3.patch
>
>
> In following method stack, seems like each mapper task will restore the
> snapshot. If we verify replication by a snapshot which has many hfiles,
> then we will take long time to restore snapshot. In our cluster, we took
> ~30min for the snapshot restoring when verify a big table.
> {code}
> Verifier.map
> |> replicatedScanner = new TableSnapshotScanner(...)
> |>
> TableSnapshotScanner.init()
>
> |-> RestoreSnapshotHelper.copySnapshotForScanner
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174246#comment-16174246 ] stack commented on HBASE-18020: --- [~alex.leblang_impala_e0fc] Hey boss... what we do here to finish up sir? Thanks. > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18020.0.patch, HBASE-18020-addendum.001.patch, > HBASE-18020.branch-1.2.001.patch, HBASE-18020.branch-1.2.002.patch, > HBASE-18020.branch-1.2.003.patch, HBASE-18020.branch-1.2.004.patch, > HBASE-18020.branch-1.2.005.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18105) [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world
[ https://issues.apache.org/jira/browse/HBASE-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174244#comment-16174244 ] stack commented on HBASE-18105: --- You ok here [~easyliangjob]? Do you need more sir? > [AMv2] Split/Merge need cleanup; currently they diverge and do not fully > embrace AMv2 world > --- > > Key: HBASE-18105 > URL: https://issues.apache.org/jira/browse/HBASE-18105 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack > Fix For: 2.0.0 > > > Region Split and Merge work on the new AMv2 but they work differently. This > issue is about bringing them back together and fully embracing the AMv2 > program. > They both have issues mostly the fact that they carry around baggage no > longer necessary in the new world of assignment. > Here are some of the items: > Split and Merge metadata modifications are done by the Master now but we have > vestige of Split/Merge on RS still; e.g. when we SPLIT, we ask the Master > which asks the RS, which turns around, and asks the Master to run the > operation. Fun. MERGE is all done Master-side. > > Clean this up. Remove asking RS to run SPLIT and remove RegionMergeRequest, > etc. on RS-side. Also remove PONR. We don’t Points-Of-No-Return now we are up > on Pv2. Remove all calls in Interfaces; they are unused. Make RS still able > to detect when split, but have it be a client of Master like anyone else. > Split is Async but does not return procId > Split is async. Doesn’t return the procId though. Merge does. Fix. Only hard > part here I think is the Admin API does not allow procid return. > Flags > Currently OFFLINE is determined by looking either at the master instance of > HTD (isOffline) and/or at the RegionState#state. Ditto for SPLIT. For MERGE, > we rely on RegionState#state. Related is a note above on how split works -- > there is a split flag in HTD when there should not be. > > TODO is move to rely on RegionState#state exclusively in Master. > From Split/Merge Procedures need finishing in > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.4b60dc1h4m1f -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18855) rewrite netty so to have proper packages
[ https://issues.apache.org/jira/browse/HBASE-18855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174237#comment-16174237 ] stack commented on HBASE-18855: --- Hmmm. I suppose it is not scheduled against a version so its fine that it is critical. > rewrite netty so to have proper packages > > > Key: HBASE-18855 > URL: https://issues.apache.org/jira/browse/HBASE-18855 > Project: HBase > Issue Type: Improvement > Components: thirdparty >Reporter: Mike Drob >Priority: Critical > > We documented the workaround for folks using surefire needing to set a system > property otherwise mini cluster won't start in HBASE-18849, but maybe we can > do better. > Can we rewrite the SO? Or patch the java code to use the correct value > instead of checking a system property for it? It should be ok to have a patch > to hardcode this sine we're telling folks to use a static value anyway. > hbase-thirdparty already has mechanisms for patching the source libs (ref. > protobuf) so the mechanics shouldn't be too complex. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18855) rewrite netty so to have proper packages
[ https://issues.apache.org/jira/browse/HBASE-18855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174236#comment-16174236 ] stack commented on HBASE-18855: --- Sure. The current route is well-trod. It is what netty community made so folks could shade and have their own .so. It is a bit of a pain, yeah, but netty-provided so will probably work a while over upgrades, etc. Waiting on a patch. Why is this 'maybe' improvement marked as critical? Good on you [~mdrob] > rewrite netty so to have proper packages > > > Key: HBASE-18855 > URL: https://issues.apache.org/jira/browse/HBASE-18855 > Project: HBase > Issue Type: Improvement > Components: thirdparty >Reporter: Mike Drob >Priority: Critical > > We documented the workaround for folks using surefire needing to set a system > property otherwise mini cluster won't start in HBASE-18849, but maybe we can > do better. > Can we rewrite the SO? Or patch the java code to use the correct value > instead of checking a system property for it? It should be ok to have a patch > to hardcode this sine we're telling folks to use a static value anyway. > hbase-thirdparty already has mechanisms for patching the source libs (ref. > protobuf) so the mechanics shouldn't be too complex. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174232#comment-16174232 ] stack commented on HBASE-16769: --- Thanks [~anoop.hbase] AC should be internal, not as a CP? Which issue added the hooks sir? > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174224#comment-16174224 ] Anoop Sam John commented on HBASE-16769: For getting the edits/cells which are getting replicated, there are ways.. Even the CP hooks around pre/post put/delete/batchMutate will get called.. The use of this particular CP hook (preReplicateLogEntries) is in AccessController. ACL was missing for the replicate APIs and that jira added this new hook. For that , there is no need for these edits any way.. Still as a general way, we passed them. Now just to call the CP we should not be making POJOs from PB objects. So we will need these hooks until our ACL works with CP impl way. There is no other go. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18109) Assign system tables first (priority)
[ https://issues.apache.org/jira/browse/HBASE-18109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174226#comment-16174226 ] stack commented on HBASE-18109: --- See[https://issues.apache.org/jira/browse/HBASE-14190?focusedCommentId=16067981&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16067981 ]from HBASE-14190, the branch-1 issue for assigning system tables ahead of user-space tables. _If I am not wrong, for system tables expect meta we still share logs with user regions, so even if we have some logic to assign system tables first, we still need wait log splitting done and we can not guarantee logs of system tables are split first. So there may be a blocking task that we should make multi-log feature having priority and write entries of system table in the highest priority logs. Thanks._ > Assign system tables first (priority) > - > > Key: HBASE-18109 > URL: https://issues.apache.org/jira/browse/HBASE-18109 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Yi Liang >Priority: Critical > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18109-V1.patch, HBASE-18109-V2.patch > > > Need this for stuff like the RSGroup table, etc. Assign these ahead of > user-space regions. > From 'Handle sys table assignment first (e.g. acl, namespace, rsgroup); > currently only hbase:meta is first.' of > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.oefcyphs0v0x -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18651: -- Attachment: HBASE-18651.master.006.patch As reviews. > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18651: -- Status: Patch Available (was: Open) > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174201#comment-16174201 ] Hudson commented on HBASE-16478: FAILURE: Integrated in Jenkins build HBase-2.0 #549 (See [https://builds.apache.org/job/HBase-2.0/549/]) Revert "HBASE-16478 Rename WALKey in PB to WALEdit This is a rebase of (stack: rev e294dcd80d2f833b6b8d2ebb088c1ccd3545212a) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSink.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALKey.java * (edit) hbase-protocol-shaded/src/main/protobuf/WAL.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogReader.java > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18160) Fix incorrect logic in FilterList.filterKeyValue
[
https://issues.apache.org/jira/browse/HBASE-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-18160:
-
Attachment: HBASE-18160.v7.patch
Simplify the AND logic for SEEK_NEXT_USING_HINT (Addresed Anoop's comment)
> Fix incorrect logic in FilterList.filterKeyValue
> -
>
> Key: HBASE-18160
> URL: https://issues.apache.org/jira/browse/HBASE-18160
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
> Attachments: filter-and-map.txt, filter-or-map.txt,
> HBASE-18160.branch-1.1.v1.patch, HBASE-18160.branch-1.v1.patch,
> HBASE-18160.v1.patch, HBASE-18160.v2.patch, HBASE-18160.v2.patch,
> HBASE-18160.v3.patch, HBASE-18160.v4.patch, HBASE-18160.v5.patch,
> HBASE-18160.v6.patch, HBASE-18160.v6.patch, HBASE-18160.v7.patch
>
>
> As HBASE-17678 said, there are two problems in FilterList.filterKeyValue
> implementation:
> 1. FilterList did not consider INCLUDE_AND_SEEK_NEXT_ROW case( seems like
> INCLUDE_AND_SEEK_NEXT_ROW is a newly added case, and the dev forgot to
> consider FilterList), So if a user use INCLUDE_AND_SEEK_NEXT_ROW in his own
> Filter and wrapped by a FilterList, it'll throw an
> IllegalStateException("Received code is not valid.").
> 2. For FilterList with MUST_PASS_ONE, if filter-A in filter list return
> INCLUDE and filter-B in filter list return INCLUDE_AND_NEXT_COL, the
> FilterList will return INCLUDE_AND_NEXT_COL finally. According to the
> mininal step rule , It's incorrect. (filter list with MUST_PASS_ONE choose
> the mininal step among filters in filter list. Let's call it: The Mininal
> Step Rule).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18807) Remove PB references from Observers for Quotas
[ https://issues.apache.org/jira/browse/HBASE-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18807: --- Attachment: HBASE-18807.002.branch-2.patch .002 I believe I've gotten all of the tests passing again. Still need to address the stability concerns Anoop mentioned, as well as do some general cleanup (logging, comments) > Remove PB references from Observers for Quotas > -- > > Key: HBASE-18807 > URL: https://issues.apache.org/jira/browse/HBASE-18807 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18807.001.branch-2.patch, > HBASE-18807.002.branch-2.patch > > > Break-out from the parent: > Same idea, just applied to the Observer methods for pre/post quota > operations. Requires changes to MasterQuotaManager and the QuotaSettings > implementations as some business logic is written on the PB objects directly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13844) Move static helper methods from KeyValue into CellUtils
[ https://issues.apache.org/jira/browse/HBASE-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174116#comment-16174116 ] Chia-Ping Tsai commented on HBASE-13844: [~andy7904] Would you please put ur patch to review board? Thanks. > Move static helper methods from KeyValue into CellUtils > --- > > Key: HBASE-13844 > URL: https://issues.apache.org/jira/browse/HBASE-13844 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Andy Yang >Priority: Minor > Labels: beginner > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-13844.1.patch, HBASE-13844.2.patch, > HBASE-13844.3.patch, HBASE-13844.branch-2.v0.patch, > HBASE-13844.branch-2.v1.patch > > > Add KeyValue.parseColumn() to CellUtils (also any other public static helper) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-13844) Move static helper methods from KeyValue into CellUtils
[ https://issues.apache.org/jira/browse/HBASE-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-13844: --- Fix Version/s: (was: 1.4.0) (was: 2.0.0) 2.0.0-beta-1 > Move static helper methods from KeyValue into CellUtils > --- > > Key: HBASE-13844 > URL: https://issues.apache.org/jira/browse/HBASE-13844 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Andy Yang >Priority: Minor > Labels: beginner > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-13844.1.patch, HBASE-13844.2.patch, > HBASE-13844.3.patch, HBASE-13844.branch-2.v0.patch, > HBASE-13844.branch-2.v1.patch > > > Add KeyValue.parseColumn() to CellUtils (also any other public static helper) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18823) Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
[ https://issues.apache.org/jira/browse/HBASE-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174114#comment-16174114 ] Chia-Ping Tsai commented on HBASE-18823: Thanks for the reviews. [~appy] > Apply RegionInfo to MasterObserver/RegionObserver/WALObserver > - > > Key: HBASE-18823 > URL: https://issues.apache.org/jira/browse/HBASE-18823 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Labels: incompatible > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18823.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18823) Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
[ https://issues.apache.org/jira/browse/HBASE-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18823: - Resolution: Fixed Status: Resolved (was: Patch Available) > Apply RegionInfo to MasterObserver/RegionObserver/WALObserver > - > > Key: HBASE-18823 > URL: https://issues.apache.org/jira/browse/HBASE-18823 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Labels: incompatible > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18823.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18823) Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
[ https://issues.apache.org/jira/browse/HBASE-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18823: - Labels: incompatible (was: ) > Apply RegionInfo to MasterObserver/RegionObserver/WALObserver > - > > Key: HBASE-18823 > URL: https://issues.apache.org/jira/browse/HBASE-18823 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Labels: incompatible > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18823.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18823) Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
[ https://issues.apache.org/jira/browse/HBASE-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174106#comment-16174106 ] Appy commented on HBASE-18823: -- I started doing it, but then realized that maybe someone has already done it. Looked for linked jiras to HBASE-17980 and voila... this! Thanks [~chia7712]! Since the logic is simple, the patch compiles, and the tests pass : +1 Pushing to master and branch-2. > Apply RegionInfo to MasterObserver/RegionObserver/WALObserver > - > > Key: HBASE-18823 > URL: https://issues.apache.org/jira/browse/HBASE-18823 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18823.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[ https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18786: --- Attachment: HBASE-18786-branch-1.patch HBASE-18786.patch bq. With the handling logic removed i think org.apache.hadoop.hbase.regionserver.RegionUnassigner will become obsolete Oops, yes, new patches that remove RegionUnassigner are now attached. For a new metric or some other support for proactive alerting, agree we should have a new issue to track that work. > FileNotFoundException should not be silently handled for primary region > replicas > > > Key: HBASE-18786 > URL: https://issues.apache.org/jira/browse/HBASE-18786 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Ashu Pachauri >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18786-branch-1.patch, HBASE-18786-branch-1.patch, > HBASE-18786.patch, HBASE-18786.patch > > > This is a follow up for HBASE-18186. > FileNotFoundException while scanning from a primary region replica can be > indicative of a more severe problem. Handling them silently can cause many > underlying issues go undetected. We should either > 1. Hard fail the regionserver if there is a FNFE on a primary region replica, > OR > 2. Report these exceptions as some region / server level metric so that these > can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18415) The local timeout may cause Admin to submit duplicate request
[
https://issues.apache.org/jira/browse/HBASE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174058#comment-16174058
]
Hadoop QA commented on HBASE-18415:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
52s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
53s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
58s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
28s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
0s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} hbase-client in the patch passed with JDK
v1.8.0_144. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} hbase-server-jdk1.8.0_144 with JDK v1.8.0_144
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} hbase-client in the patch passed with JDK
v1.7.0_151. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} hbase-server-jdk1.7.0_151 with JDK v1.7.0_151
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 33s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} hbase-client-jdk1.8.0_144 with JDK v1.8.0_144
generated 0 new + 13 unchanged - 13 fixed = 13 total (was 26) {color} |
| {color
[jira] [Commented] (HBASE-18855) rewrite netty so to have proper packages
[ https://issues.apache.org/jira/browse/HBASE-18855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174039#comment-16174039 ] Duo Zhang commented on HBASE-18855: --- +1. Let's do it in next release of hbase-thirdparty sir [~stack]? Thanks. > rewrite netty so to have proper packages > > > Key: HBASE-18855 > URL: https://issues.apache.org/jira/browse/HBASE-18855 > Project: HBase > Issue Type: Improvement > Components: thirdparty >Reporter: Mike Drob >Priority: Critical > > We documented the workaround for folks using surefire needing to set a system > property otherwise mini cluster won't start in HBASE-18849, but maybe we can > do better. > Can we rewrite the SO? Or patch the java code to use the correct value > instead of checking a system property for it? It should be ok to have a patch > to hardcode this sine we're telling folks to use a static value anyway. > hbase-thirdparty already has mechanisms for patching the source libs (ref. > protobuf) so the mechanics shouldn't be too complex. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174012#comment-16174012
]
Ted Yu commented on HBASE-18796:
At commit a29ea36194a18c1de415577e78ca3553818a72b5, TestMultiRespectsLimits
passes on master branch.
When switching to commit 29a3ff303799299ce42b57e85b2a2ac575dab474, the test
fails:
{code}
testMultiLimits(org.apache.hadoop.hbase.client.TestMultiRespectsLimits) Time
elapsed: 18.346 sec <<< ERROR!
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
action: testMultiLimits: 1 time, servers with issues: null
at
org.apache.hadoop.hbase.client.TestMultiRespectsLimits.testMultiLimits(TestMultiRespectsLimits.java:106)
{code}
testMultiLimits involves splitting region.
Abhishek:
Mind taking a look ?
> Admin#isTableAvailable returns incorrect result before daughter regions are
> opened
> --
>
> Key: HBASE-18796
> URL: https://issues.apache.org/jira/browse/HBASE-18796
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0
>
> Attachments: HBASE-18796.branch-1.001.patch,
> HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch,
> HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch
>
>
> Admin#isTableAvailable checks if it can getServerName for the meta entries it
> reads. During the time of split server location are added to the meta entries
> in MetaTableAccessor#splitRegion although the description of the method says
> "Does not add the location information to the daughter regions since they are
> not open yet.". At this point during the split daughter regions are not
> actually open, so we can get to a state where parent is offline, daughters
> are not yet open but isTableAvailable returns true.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18800) [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going forward
[ https://issues.apache.org/jira/browse/HBASE-18800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18800: -- Fix Version/s: 2.0.0-beta-1 > [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going > forward > -- > > Key: HBASE-18800 > URL: https://issues.apache.org/jira/browse/HBASE-18800 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 2.0.0-beta-1 > > > We've bandied this about for a while now that folks should consume hbase via > the shaded hbase-client; it should work if their needs are minimal (and if it > doesn't work, would be good to hear why). This issue is about evangelizing > the shaded hbase-client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18773) Add utility to determine if a TableName is a meta table
[
https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173965#comment-16173965
]
Hadoop QA commented on HBASE-18773:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
36s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
26s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
13s{color} | {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
13s{color} | {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 13s{color}
| {color:red} hbase-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
52s{color} | {color:red} patch has 16 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
33s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
14s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
54s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.3. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
34s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
14s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
55s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
34s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
14s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.3. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
54s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0-alpha4.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
10s{color} | {color:red} hbase-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 14s{color}
| {color:red} hbase-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{c
[jira] [Updated] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18773: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HBASE-18477 Status: Resolved (was: Patch Available) Pushed to HBASE-18477. > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Fix For: HBASE-18477 > > Attachments: > 0001-HBASE-18773-Add-utility-method-to-determine-if-a-Tab.patch, > HBASE-18773.HBASE-18477.001.patch, HBASE-18773.HBASE-18477.002.patch, > HBASE-18773.HBASE-18477.003.patch, HBASE-18773.HBASE-18477.004.patch, > HBASE-18773.HBASE-18477.005.patch, HBASE-18773.HBASE-18477.006.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173948#comment-16173948 ] stack edited comment on HBASE-16478 at 9/20/17 10:39 PM: - Ok. I reverted this patch from branch-2 and master. It made it so the protobuf in hbase-protocol could no longer parse a WAL entry. The change was not worth it in 2.0. Leaving as reopened. Took the fix version off it. was (Author: stack): Ok. I reverted this patch. It made it so the protobuf in hbase-protocol could no longer parse a WAL entry. The change was not worth it in 2.0. Leaving as reopened. Took the fix version off it. > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16478: -- Fix Version/s: (was: 2.0.0-alpha-4) > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173948#comment-16173948 ] stack commented on HBASE-16478: --- Ok. I reverted this patch. It made it so the protobuf in hbase-protocol could no longer parse a WAL entry. The change was not worth it in 2.0. Leaving as reopened. Took the fix version off it. > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18773: -- Affects Version/s: HBASE-18477 > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Attachments: > 0001-HBASE-18773-Add-utility-method-to-determine-if-a-Tab.patch, > HBASE-18773.HBASE-18477.001.patch, HBASE-18773.HBASE-18477.002.patch, > HBASE-18773.HBASE-18477.003.patch, HBASE-18773.HBASE-18477.004.patch, > HBASE-18773.HBASE-18477.005.patch, HBASE-18773.HBASE-18477.006.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173933#comment-16173933 ] Zach York commented on HBASE-18773: --- [~stack] Yeah it didn't seem to belong in hbase-common, but I wasn't quite sure where to put it. Let's address it afterwards. This is developing against the https://github.com/apache/hbase/tree/HBASE-18477 branch (feature branch). > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: > 0001-HBASE-18773-Add-utility-method-to-determine-if-a-Tab.patch, > HBASE-18773.HBASE-18477.001.patch, HBASE-18773.HBASE-18477.002.patch, > HBASE-18773.HBASE-18477.003.patch, HBASE-18773.HBASE-18477.004.patch, > HBASE-18773.HBASE-18477.005.patch, HBASE-18773.HBASE-18477.006.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18478) Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
[ https://issues.apache.org/jira/browse/HBASE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173926#comment-16173926 ] Ted Yu commented on HBASE-18478: hmm Would change to src/site/site.xml cause the test to fail ? > Allow users to remove RegionFinder from LoadBalancer calculations if no > locality possible > - > > Key: HBASE-18478 > URL: https://issues.apache.org/jira/browse/HBASE-18478 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18478.branch-1.001.patch, > HBASE-18478.master.001.patch > > > BaseLoadBalancer should have the option to remove RegionFinder from load > balancing. This provides significant cluster start time reduction for > FileSystems that do not surface locality such as Amazon S3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18773: -- Attachment: 0001-HBASE-18773-Add-utility-method-to-determine-if-a-Tab.patch Where you dev'ing against [~zyork]? I rebased your patch to apply to master but it doesn't compile because there is no define for TableName.DEFAULT_META_TABLE_NAME_STR. I can't seem to find it. Something I'm doing wrong. Thanks. > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: > 0001-HBASE-18773-Add-utility-method-to-determine-if-a-Tab.patch, > HBASE-18773.HBASE-18477.001.patch, HBASE-18773.HBASE-18477.002.patch, > HBASE-18773.HBASE-18477.003.patch, HBASE-18773.HBASE-18477.004.patch, > HBASE-18773.HBASE-18477.005.patch, HBASE-18773.HBASE-18477.006.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18478) Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
[ https://issues.apache.org/jira/browse/HBASE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173919#comment-16173919 ] Andrew Purtell commented on HBASE-18478: Ok I switched to a different dev box (Linux) and can repro, but the failure happens at bc790fe26acb9449afd7b3d316e6661933a86b40 on master which is the commit before this one, so you're barking up the wrong tree Ted. > Allow users to remove RegionFinder from LoadBalancer calculations if no > locality possible > - > > Key: HBASE-18478 > URL: https://issues.apache.org/jira/browse/HBASE-18478 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18478.branch-1.001.patch, > HBASE-18478.master.001.patch > > > BaseLoadBalancer should have the option to remove RegionFinder from load > balancing. This provides significant cluster start time reduction for > FileSystems that do not surface locality such as Amazon S3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18478) Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
[ https://issues.apache.org/jira/browse/HBASE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173907#comment-16173907 ] Ted Yu commented on HBASE-18478: Here is the version of Java on Linux where I can reproduce the test failure: java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) Flaky test board led me to this JIRA since the commit was the first build where the test failed - and it has been failing consistently on Apache Jenkins. > Allow users to remove RegionFinder from LoadBalancer calculations if no > locality possible > - > > Key: HBASE-18478 > URL: https://issues.apache.org/jira/browse/HBASE-18478 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18478.branch-1.001.patch, > HBASE-18478.master.001.patch > > > BaseLoadBalancer should have the option to remove RegionFinder from load > balancing. This provides significant cluster start time reduction for > FileSystems that do not surface locality such as Amazon S3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173903#comment-16173903 ] stack commented on HBASE-18773: --- Ugh. My fault. Sorry for the neglect [~zyork]. Let me commit this though I have issue w/ it. Issues can be addressed in follow-on. IMO, ReadReplicaClustersTableNameUtil does not belong in hbase-common. ReadReplicaClusters is an exotic feature that should be well-encapsulated minimally inside a package or in its own module even (might not be enough substance to warrant its own module). Perhaps there is another place this class might go? No biggie. > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18773.HBASE-18477.001.patch, > HBASE-18773.HBASE-18477.002.patch, HBASE-18773.HBASE-18477.003.patch, > HBASE-18773.HBASE-18477.004.patch, HBASE-18773.HBASE-18477.005.patch, > HBASE-18773.HBASE-18477.006.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18478) Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
[ https://issues.apache.org/jira/browse/HBASE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173883#comment-16173883 ] Andrew Purtell commented on HBASE-18478: TestMultiRespectsLimits passes for me repeatedly. Test JRE is 8u131. Checked with master, branch-2, and branch-1. Not sure how that failure could be related to this change. I'm guessing something else is going on. Can you post detail sufficient for reproduction [~ted_yu]? I wouldn't have committed if the branch-1 suite had a failure due to this change, FWIW > Allow users to remove RegionFinder from LoadBalancer calculations if no > locality possible > - > > Key: HBASE-18478 > URL: https://issues.apache.org/jira/browse/HBASE-18478 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18478.branch-1.001.patch, > HBASE-18478.master.001.patch > > > BaseLoadBalancer should have the option to remove RegionFinder from load > balancing. This provides significant cluster start time reduction for > FileSystems that do not surface locality such as Amazon S3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18422) Fix TestRegionRebalancing
[ https://issues.apache.org/jira/browse/HBASE-18422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173850#comment-16173850 ] Vladimir Rodionov commented on HBASE-18422: --- Will resume once I finish 2 remaining blockers in B&R, [~elserj] > Fix TestRegionRebalancing > - > > Key: HBASE-18422 > URL: https://issues.apache.org/jira/browse/HBASE-18422 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18422-v1.patch, HBASE-18422-v2.patch, > HBASE-18422-v3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18422) Fix TestRegionRebalancing
[ https://issues.apache.org/jira/browse/HBASE-18422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173807#comment-16173807 ] Josh Elser commented on HBASE-18422: This one still on your radar, [~vrodionov]? I know you have started on the Distcp fixes for B&R as well. > Fix TestRegionRebalancing > - > > Key: HBASE-18422 > URL: https://issues.apache.org/jira/browse/HBASE-18422 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18422-v1.patch, HBASE-18422-v2.patch, > HBASE-18422-v3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173800#comment-16173800
]
stack commented on HBASE-17399:
---
Dump in here examples of how the issue is fixed by the patch please [~chunhao]
Thank you.
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch,
> HBASE-17399.branch-1.3.v1.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[
https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173793#comment-16173793
]
stack commented on HBASE-16769:
---
Thanks for working on this ugly one [~anoop.hbase]
What about this:
default void preReplicateLogEntries(final
ObserverContext ctx)
96throws IOException {
97}
And for the post.
Are these invocations any good if they don't pass the actual Edits that are
about to be shipped?
How about just stripping them altogether and pointing folks at
ReplicationEndpoint instead? There they get all the edits and do what they
wilst with them. It even has simple filter function. It is problematic in that
WALEntryFilter, the Interface, operates on Entry, WALEntry and WALEdit POJOs
that are marked Audience Private. We need to fix this but that is a different,
non-CP issue. TODO.
Otherwise, patch looks great. Any other PBs to purge from CPs?
> Deprecate/remove PB references from MasterObserver and RegionServerObserver
> ---
>
> Key: HBASE-16769
> URL: https://issues.apache.org/jira/browse/HBASE-16769
> Project: HBase
> Issue Type: Bug
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-16769.patch, HBASE-16769_V2.patch
>
>
> This is effectively a sub-task for HBASE-15174.
> CP Methods
> MasterObserver
> preListSnapshot
> postListSnapshot
> preSnapshot
> postSnapshot
> preCloneSnapshot
> postCloneSnapshot
> preRestoreSnapshot
> postRestoreSnapshot
> preDeleteSnapshot
> postDeleteSnapshot
>
> preSetUserQuota
> postSetUserQuota
> preSetUserQuota
> postSetUserQuota
> preSetUserQuota
> postSetUserQuota
> preSetTableQuota
> postSetTableQuota
> preSetNamespaceQuota
> postSetNamespaceQuota
>
> RegionServerObserver
> preReplicateLogEntries
> postReplicateLogEntries
> Note : This issue not handling Quota related CPs. Same is handled by a
> subtask here HBase-18807
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18853) hbase-protocol-shaded includes protobuf (since we moved to hbase-thirdparty)
[ https://issues.apache.org/jira/browse/HBASE-18853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173785#comment-16173785 ] Hudson commented on HBASE-18853: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3748 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3748/]) HBASE-18853 hbase-protocol-shaded includes protobuf (since we moved to (stack: rev e3896cfcc37bc157c75200592e2eaa6be7107856) * (edit) hbase-protocol-shaded/pom.xml > hbase-protocol-shaded includes protobuf (since we moved to hbase-thirdparty) > > > Key: HBASE-18853 > URL: https://issues.apache.org/jira/browse/HBASE-18853 > Project: HBase > Issue Type: Bug > Components: thirdparty >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18853.master.001.patch, > HBASE-18853.master.002.patch > > > I didn't follow my own comment when I added hbase-thirdparty > hbase-shaded-protobuf to hbase-shaded-protocol as a dependency instead of > original protobuf. It meant that our jar had protobuf bundled up inside it. > Change is small, and I don't think we were actually using the non-shaded > protobuf anywhere, but if anyone was, it'd make for interesting issue to > debug. Let me see if it breaks anything... > (Found by our [~vickyuec] on internal deploy...) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18478) Allow users to remove RegionFinder from LoadBalancer calculations if no locality possible
[ https://issues.apache.org/jira/browse/HBASE-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173781#comment-16173781 ] Ted Yu commented on HBASE-18478: This test fails consistently after this went in: https://builds.apache.org/job/HBASE-Flaky-Tests/21513/testReport/junit/org.apache.hadoop.hbase.client/TestMultiRespectsLimits/testMultiLimits/ Zach: Can you check ? > Allow users to remove RegionFinder from LoadBalancer calculations if no > locality possible > - > > Key: HBASE-18478 > URL: https://issues.apache.org/jira/browse/HBASE-18478 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18478.branch-1.001.patch, > HBASE-18478.master.001.patch > > > BaseLoadBalancer should have the option to remove RegionFinder from load > balancing. This provides significant cluster start time reduction for > FileSystems that do not surface locality such as Amazon S3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18651: -- Status: Open (was: Patch Available) > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18850) [c++] Update BUILDING and README
[ https://issues.apache.org/jira/browse/HBASE-18850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-18850: -- Attachment: hbase-18850_v1.patch > [c++] Update BUILDING and README > > > Key: HBASE-18850 > URL: https://issues.apache.org/jira/browse/HBASE-18850 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: HBASE-14850 > > Attachments: hbase-18850_v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173759#comment-16173759
]
Hadoop QA commented on HBASE-17399:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
18s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
7s{color} | {color:red} The patch generated 1 new + 322 unchanged - 4 fixed =
323 total (was 326) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
2s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
42s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
47s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b3a2a00 |
| JIRA Issue | HBASE-17399 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888109/HBASE-17399.branch-1.3.v1.patch
|
| Optional Tests | asflicense shadedjars rubocop ruby_lint |
| uname | Linux fb7461a1d298 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-1.3 / ee6131f |
| rubocop | v0.50.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/8716/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8716/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch,
> HBASE-17399.branch-1.3.v1.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18853) hbase-protocol-shaded includes protobuf (since we moved to hbase-thirdparty)
[ https://issues.apache.org/jira/browse/HBASE-18853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173749#comment-16173749 ] Hudson commented on HBASE-18853: FAILURE: Integrated in Jenkins build HBase-2.0 #547 (See [https://builds.apache.org/job/HBase-2.0/547/]) HBASE-18853 hbase-protocol-shaded includes protobuf (since we moved to (stack: rev 76199a338dde5a0962a4b206d99a9baa371f9829) * (edit) hbase-protocol-shaded/pom.xml > hbase-protocol-shaded includes protobuf (since we moved to hbase-thirdparty) > > > Key: HBASE-18853 > URL: https://issues.apache.org/jira/browse/HBASE-18853 > Project: HBase > Issue Type: Bug > Components: thirdparty >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18853.master.001.patch, > HBASE-18853.master.002.patch > > > I didn't follow my own comment when I added hbase-thirdparty > hbase-shaded-protobuf to hbase-shaded-protocol as a dependency instead of > original protobuf. It meant that our jar had protobuf bundled up inside it. > Change is small, and I don't think we were actually using the non-shaded > protobuf anywhere, but if anyone was, it'd make for interesting issue to > debug. Let me see if it breaks anything... > (Found by our [~vickyuec] on internal deploy...) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15154) Master puts up a blockcache instance
[ https://issues.apache.org/jira/browse/HBASE-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173745#comment-16173745 ] stack commented on HBASE-15154: --- I set this as blocker. HBASE-18511 is where we disable master carrying regions by default. The boolean flag hbase.balancer.tablesOnMaster might help here (see release note in HBASE-18511). > Master puts up a blockcache instance > > > Key: HBASE-15154 > URL: https://issues.apache.org/jira/browse/HBASE-15154 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Blocker > Fix For: 2.0.0 > > > Master is putting up a blockcache instance. It shouldn't. This issue comes of > the unification of Master and RegionServer where Master is a subclass of > RegionServer. Our [~jmspaggi] found out the hard way today when he tried to > bring up a cluster with a large offheap cache only the master member had been > given less memory. > Marking critical rather than blocker because there is a means of configuring > your way out of this little entanglement; set hbase.bucketcache.size to zero > in the master config only. > If you want to set direct memory for regionservers only and not on the > master, you should do the following: > Set HBASE_OFFHEAPSIZE=0G > ... and then turn it on for RegionServers only: > HBASE_REGIONSERVER_OPTS="HBASE_REGIONSERVER_OPTS > -XX:MaxDirectMemorySize=XXXG" where XXX is size of the direct memory you want > to allocate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15154) Master puts up a blockcache instance
[ https://issues.apache.org/jira/browse/HBASE-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15154: -- Priority: Blocker (was: Critical) > Master puts up a blockcache instance > > > Key: HBASE-15154 > URL: https://issues.apache.org/jira/browse/HBASE-15154 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Blocker > Fix For: 2.0.0 > > > Master is putting up a blockcache instance. It shouldn't. This issue comes of > the unification of Master and RegionServer where Master is a subclass of > RegionServer. Our [~jmspaggi] found out the hard way today when he tried to > bring up a cluster with a large offheap cache only the master member had been > given less memory. > Marking critical rather than blocker because there is a means of configuring > your way out of this little entanglement; set hbase.bucketcache.size to zero > in the master config only. > If you want to set direct memory for regionservers only and not on the > master, you should do the following: > Set HBASE_OFFHEAPSIZE=0G > ... and then turn it on for RegionServers only: > HBASE_REGIONSERVER_OPTS="HBASE_REGIONSERVER_OPTS > -XX:MaxDirectMemorySize=XXXG" where XXX is size of the direct memory you want > to allocate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15154) Master puts up a blockcache instance
[ https://issues.apache.org/jira/browse/HBASE-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15154: -- Fix Version/s: 2.0.0 > Master puts up a blockcache instance > > > Key: HBASE-15154 > URL: https://issues.apache.org/jira/browse/HBASE-15154 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Blocker > Fix For: 2.0.0 > > > Master is putting up a blockcache instance. It shouldn't. This issue comes of > the unification of Master and RegionServer where Master is a subclass of > RegionServer. Our [~jmspaggi] found out the hard way today when he tried to > bring up a cluster with a large offheap cache only the master member had been > given less memory. > Marking critical rather than blocker because there is a means of configuring > your way out of this little entanglement; set hbase.bucketcache.size to zero > in the master config only. > If you want to set direct memory for regionservers only and not on the > master, you should do the following: > Set HBASE_OFFHEAPSIZE=0G > ... and then turn it on for RegionServers only: > HBASE_REGIONSERVER_OPTS="HBASE_REGIONSERVER_OPTS > -XX:MaxDirectMemorySize=XXXG" where XXX is size of the direct memory you want > to allocate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173727#comment-16173727 ] stack commented on HBASE-18840: --- bq. Are the region infos not updated on region merge/splits? Split and merges get a .regioninfo. The parent at least in the split case will still be around so you'll need to figure which to favor reconstructing the meta. You are dev'ing against branch-1 I take it? Here is a hack from a million years ago to do similar HBASE-1867 > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChunHao updated HBASE-17399:
Status: Patch Available (was: Open)
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch,
> HBASE-17399.branch-1.3.v1.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChunHao updated HBASE-17399:
Status: Open (was: Patch Available)
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch,
> HBASE-17399.branch-1.3.v1.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChunHao updated HBASE-17399:
Attachment: HBASE-17399.branch-1.3.v1.patch
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch,
> HBASE-17399.branch-1.3.v1.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18624) Added support for clearing BlockCache based on table name
[
https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173684#comment-16173684
]
Ted Yu commented on HBASE-18624:
{code}
+ public void clearBlockCache(final TableName tableName) throws IOException {
{code}
How about changing the return type to integer which represents the number of
regions who cache is not cleared ?
This would give caller some idea how effective the call is.
> Added support for clearing BlockCache based on table name
> -
>
> Key: HBASE-18624
> URL: https://issues.apache.org/jira/browse/HBASE-18624
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18624.branch-1.001.patch,
> HBASE-18624.master.001.patch, HBASE-18624.master.002.patch
>
>
> Bulk loading the primary HBase cluster triggers a lot of compactions
> resulting in archival/ creation
> of multiple HFiles. This process will cause a lot of items to become stale in
> replica’s BlockCache.
> This patch will help users to clear the block cache for a given table by
> either using shell or API.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18624) Added support for clearing BlockCache based on table name
[
https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173676#comment-16173676
]
Hadoop QA commented on HBASE-18624:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
8s{color} | {color:red} The patch generated 8 new + 356 unchanged - 1 fixed =
364 total (was 357) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m
5s{color} | {color:red} The patch generated 4 new + 728 unchanged - 0 fixed =
732 total (was 728) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
39s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
56s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicen
[jira] [Commented] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173665#comment-16173665
]
Hadoop QA commented on HBASE-17399:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
13s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
8s{color} | {color:red} The patch generated 5 new + 323 unchanged - 3 fixed =
328 total (was 326) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
3s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b3a2a00 |
| JIRA Issue | HBASE-17399 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888095/HBASE-17399.branch-1.3.v0.patch
|
| Optional Tests | asflicense shadedjars rubocop ruby_lint |
| uname | Linux 9a4910caa4b9 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-1.3 / ee6131f |
| rubocop | v0.50.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/8715/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8715/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[
https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173593#comment-16173593
]
Hadoop QA commented on HBASE-16769:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red}499m
46s{color} | {color:red} Docker failed to build yetus/hbase:5d60123. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-16769 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888034/HBASE-16769_V2.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8712/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Deprecate/remove PB references from MasterObserver and RegionServerObserver
> ---
>
> Key: HBASE-16769
> URL: https://issues.apache.org/jira/browse/HBASE-16769
> Project: HBase
> Issue Type: Bug
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-16769.patch, HBASE-16769_V2.patch
>
>
> This is effectively a sub-task for HBASE-15174.
> CP Methods
> MasterObserver
> preListSnapshot
> postListSnapshot
> preSnapshot
> postSnapshot
> preCloneSnapshot
> postCloneSnapshot
> preRestoreSnapshot
> postRestoreSnapshot
> preDeleteSnapshot
> postDeleteSnapshot
>
> preSetUserQuota
> postSetUserQuota
> preSetUserQuota
> postSetUserQuota
> preSetUserQuota
> postSetUserQuota
> preSetTableQuota
> postSetTableQuota
> preSetNamespaceQuota
> postSetNamespaceQuota
>
> RegionServerObserver
> preReplicateLogEntries
> postReplicateLogEntries
> Note : This issue not handling Quota related CPs. Same is handled by a
> subtask here HBase-18807
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChunHao updated HBASE-17399:
Attachment: HBASE-17399.branch-1.3.v0.patch
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChunHao updated HBASE-17399:
Status: Patch Available (was: Open)
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15154) Master puts up a blockcache instance
[
https://issues.apache.org/jira/browse/HBASE-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173606#comment-16173606
]
Biju Nair commented on HBASE-15154:
---
bq. If you want to set direct memory for regionservers only and not on the
master, you should do the following:
Set HBASE_OFFHEAPSIZE=0G
We also will need to have a different {{hbase-site.xml}} for {{master}} which
doesn't enable bucketCache. no?
> Master puts up a blockcache instance
>
>
> Key: HBASE-15154
> URL: https://issues.apache.org/jira/browse/HBASE-15154
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
>
> Master is putting up a blockcache instance. It shouldn't. This issue comes of
> the unification of Master and RegionServer where Master is a subclass of
> RegionServer. Our [~jmspaggi] found out the hard way today when he tried to
> bring up a cluster with a large offheap cache only the master member had been
> given less memory.
> Marking critical rather than blocker because there is a means of configuring
> your way out of this little entanglement; set hbase.bucketcache.size to zero
> in the master config only.
> If you want to set direct memory for regionservers only and not on the
> master, you should do the following:
> Set HBASE_OFFHEAPSIZE=0G
> ... and then turn it on for RegionServers only:
> HBASE_REGIONSERVER_OPTS="HBASE_REGIONSERVER_OPTS
> -XX:MaxDirectMemorySize=XXXG" where XXX is size of the direct memory you want
> to allocate.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18624) Added support for clearing BlockCache based on table name
[ https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173569#comment-16173569 ] Ajay Jadhav commented on HBASE-18624: - Regarding the exception: The NotServingRegionException can only happen in case the meta table is corrupted resulting in providing an incorrect location for a region. This may also happen due to some region operation in progress (split) etc which meta is not aware of. In case I rethrow the exception it will stop the processing of all the subsequent regions due to one bad entry in meta. Also, this will be one-off case due to which I'm just logging the exception. I'll move HBaseRpcController out of for loop. Thanks. > Added support for clearing BlockCache based on table name > - > > Key: HBASE-18624 > URL: https://issues.apache.org/jira/browse/HBASE-18624 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.0 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18624.branch-1.001.patch, > HBASE-18624.master.001.patch, HBASE-18624.master.002.patch > > > Bulk loading the primary HBase cluster triggers a lot of compactions > resulting in archival/ creation > of multiple HFiles. This process will cause a lot of items to become stale in > replica’s BlockCache. > This patch will help users to clear the block cache for a given table by > either using shell or API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18833) Ensure precommit personality is up to date on all active branches
[ https://issues.apache.org/jira/browse/HBASE-18833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173527#comment-16173527 ] Hudson commented on HBASE-18833: FAILURE: Integrated in Jenkins build HBASE-14070.HLC #431 (See [https://builds.apache.org/job/HBASE-14070.HLC/431/]) HBASE-18833 Ensure precommit personality is up to date on all active (busbey: rev 93d87d17e441cfee63fc2827a4e974e4550fa9d2) * (edit) dev-support/hbase-personality.sh > Ensure precommit personality is up to date on all active branches > - > > Key: HBASE-18833 > URL: https://issues.apache.org/jira/browse/HBASE-18833 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.4.0, HBASE-14850, 1.3.2, 1.5.0, 1.2.7, HBASE-14070, > 1.1.13, 2.0.0-alpha-4, HBASE-15631, HBASE-18477 > > Attachments: HBASE-18833-branch-1.v0.patch, > HBASE-18833-branch-1.v1.patch, HBASE-18833-branch-1.v2.patch, > HBASE-18833-branch-2.v0.patch, HBASE-18833-branch-2.v1.patch, > HBASE-18833-branch-2.v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18624) Added support for clearing BlockCache based on table name
[
https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173521#comment-16173521
]
Ted Yu commented on HBASE-18624:
{code}
+ } catch (NotServingRegionException e) {
+if (LOG.isDebugEnabled()) {
+ LOG.debug("Trying to clear block cache for " + pair.getFirst() + ":
" +
+StringUtils.stringifyException(e));
+}
{code}
Why is exception swallowed ?
{code}
+ private void clearBlockCache(final ServerName sn, final HRegionInfo hri)
throws IOException {
+HBaseRpcController controller = rpcControllerFactory.newController();
+AdminService.BlockingInterface admin = this.connection.getAdmin(sn);
{code}
HBaseRpcController is created for every region. Please lift this out of the
loop.
> Added support for clearing BlockCache based on table name
> -
>
> Key: HBASE-18624
> URL: https://issues.apache.org/jira/browse/HBASE-18624
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0, 1.3.0
>Reporter: Ajay Jadhav
>Assignee: Ajay Jadhav
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18624.branch-1.001.patch,
> HBASE-18624.master.001.patch, HBASE-18624.master.002.patch
>
>
> Bulk loading the primary HBase cluster triggers a lot of compactions
> resulting in archival/ creation
> of multiple HFiles. This process will cause a lot of items to become stale in
> replica’s BlockCache.
> This patch will help users to clear the block cache for a given table by
> either using shell or API.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173472#comment-16173472 ] Anoop Sam John edited comment on HBASE-16769 at 9/20/17 4:41 PM: - Ping [~ram_krish], [~stack] for reviews was (Author: anoop.hbase): Ping [~ram_krish], [~st...@gmail.com] for reviews > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173472#comment-16173472 ] Anoop Sam John commented on HBASE-16769: Ping [~ram_krish], [~st...@gmail.com] for reviews > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173469#comment-16173469 ] Anoop Sam John commented on HBASE-16769: Thanks for the review.. Will fix... Ya I was trying to fix an existing issue where the op to check for Snapshot Support was doing CP calls. But missed this condition check. Tks for the careful read > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18523) FilterList should break if the filter succeeds (row passes) in case of MUST_PASS_ONE and if the filter fails (row is skipped) for MUST_PASS_ALL
[ https://issues.apache.org/jira/browse/HBASE-18523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173467#comment-16173467 ] Deon Huang commented on HBASE-18523: Just notice that branch-2 has some potential optimization for filterRowKey(). While filterRow() is already an example of the description optimization. Should we add affect version for branch-2, branch-1.4, branch-1.5? May I take this issue? Thanks > FilterList should break if the filter succeeds (row passes) in case of > MUST_PASS_ONE and if the filter fails (row is skipped) for MUST_PASS_ALL > --- > > Key: HBASE-18523 > URL: https://issues.apache.org/jira/browse/HBASE-18523 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1, 0.98.24, 1.2.6, 1.1.12 >Reporter: Akshita Malhotra > Labels: beginner, patch > > FilterList should break if the filter succeeds (row passes) in case of > MUST_PASS_ONE and if the filter fails (row is skipped) for MUST_PASS_ALL -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18855) rewrite netty so to have proper packages
[ https://issues.apache.org/jira/browse/HBASE-18855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18855: -- Priority: Critical (was: Major) > rewrite netty so to have proper packages > > > Key: HBASE-18855 > URL: https://issues.apache.org/jira/browse/HBASE-18855 > Project: HBase > Issue Type: Improvement > Components: thirdparty >Reporter: Mike Drob >Priority: Critical > > We documented the workaround for folks using surefire needing to set a system > property otherwise mini cluster won't start in HBASE-18849, but maybe we can > do better. > Can we rewrite the SO? Or patch the java code to use the correct value > instead of checking a system property for it? It should be ok to have a patch > to hardcode this sine we're telling folks to use a static value anyway. > hbase-thirdparty already has mechanisms for patching the source libs (ref. > protobuf) so the mechanics shouldn't be too complex. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18855) rewrite netty so to have proper packages
Mike Drob created HBASE-18855: - Summary: rewrite netty so to have proper packages Key: HBASE-18855 URL: https://issues.apache.org/jira/browse/HBASE-18855 Project: HBase Issue Type: Improvement Components: thirdparty Reporter: Mike Drob We documented the workaround for folks using surefire needing to set a system property otherwise mini cluster won't start in HBASE-18849, but maybe we can do better. Can we rewrite the SO? Or patch the java code to use the correct value instead of checking a system property for it? It should be ok to have a patch to hardcode this sine we're telling folks to use a static value anyway. hbase-thirdparty already has mechanisms for patching the source libs (ref. protobuf) so the mechanics shouldn't be too complex. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18833) Ensure precommit personality is up to date on all active branches
[ https://issues.apache.org/jira/browse/HBASE-18833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173425#comment-16173425 ] Hudson commented on HBASE-18833: FAILURE: Integrated in Jenkins build HBase-2.0 #546 (See [https://builds.apache.org/job/HBase-2.0/546/]) HBASE-18833 Ensure precommit personality is up to date on all active (busbey: rev ea610cfcf7416e0dcdf04f93e968ba91d07e2b76) * (edit) dev-support/hbase-personality.sh > Ensure precommit personality is up to date on all active branches > - > > Key: HBASE-18833 > URL: https://issues.apache.org/jira/browse/HBASE-18833 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.4.0, HBASE-14850, 1.3.2, 1.5.0, 1.2.7, HBASE-14070, > 1.1.13, 2.0.0-alpha-4, HBASE-15631, HBASE-18477 > > Attachments: HBASE-18833-branch-1.v0.patch, > HBASE-18833-branch-1.v1.patch, HBASE-18833-branch-1.v2.patch, > HBASE-18833-branch-2.v0.patch, HBASE-18833-branch-2.v1.patch, > HBASE-18833-branch-2.v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18624) Added support for clearing BlockCache based on table name
[ https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173414#comment-16173414 ] Ajay Jadhav commented on HBASE-18624: - Addressed comments in the latest patch. > Added support for clearing BlockCache based on table name > - > > Key: HBASE-18624 > URL: https://issues.apache.org/jira/browse/HBASE-18624 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.0 >Reporter: Ajay Jadhav >Assignee: Ajay Jadhav > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18624.branch-1.001.patch, > HBASE-18624.master.001.patch, HBASE-18624.master.002.patch > > > Bulk loading the primary HBase cluster triggers a lot of compactions > resulting in archival/ creation > of multiple HFiles. This process will cause a lot of items to become stale in > replica’s BlockCache. > This patch will help users to clear the block cache for a given table by > either using shell or API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
