[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174389#comment-16174389 ] Abhishek Singh Chouhan commented on HBASE-18796: Looking into it. Thanks [~tedyu]. > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18823) Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
[ https://issues.apache.org/jira/browse/HBASE-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174412#comment-16174412 ] Hudson commented on HBASE-18823: FAILURE: Integrated in Jenkins build HBase-2.0 #550 (See [https://builds.apache.org/job/HBase-2.0/550/]) HBASE-18823 Apply RegionInfo to (appy: rev 0eab16fde4dfe5eee47aac7f7ba87f71e04e3cca) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestEnableTable.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/WALObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ExampleMasterObserverWithMetrics.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/CoprocessorWhitelistMasterObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverForAddingMutationsFromCoprocessors.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/BaseTestHBaseFsck.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SampleRegionWALObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithRemove.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorMetrics.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java > Apply RegionInfo to MasterObserver/RegionObserver/WALObserver > - > > Key: HBASE-18823 > URL: https://issues.apache.org/jira/browse/HBASE-18823 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Labels: incompatible > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18823.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174426#comment-16174426 ] Anoop Sam John commented on HBASE-16769: bq.AC should be internal, not as a CP? That would be ideal.. But that would be a big work. So as of now this hook is needed. bq.Which issue added the hooks sir? HBASE-12916 > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[
https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174463#comment-16174463
]
Hadoop QA commented on HBASE-18786:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
26s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
26s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 42s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}380m 14s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 3m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}433m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestClientOperationInterrupt |
| | hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort |
| | hadoop
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174489#comment-16174489
]
Anoop Sam John commented on HBASE-13346:
Filter.java
Why new filterCell method missing from here?! We need add here
No need to do any extra things for filterKeyValue in FilterBase (than what we
may have today)
FilterList#filterKeyValue() - This calls filterKeyValue() on individual
filters. This also we have to change.. Within our code base, we have to call
the new method only. The default impl of the new method has to call the old
method (This has to be done in Filter base class). This is for BC for custom
filters. Users even have directly extended Filter class. That is why we have
to add there only with a def impl (We are on JDK 1.8+)
Import
Filter.ReturnCode code = filter.filterKeyValue(c);
Change here also to filterCell? Any way the def impl would be calling old
method and if the Import is having new code (new jar) Filter class also would
be new.
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17782) Extend IdReadWriteLock to support using both weak and soft reference
[
https://issues.apache.org/jira/browse/HBASE-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174535#comment-16174535
]
Yechao Chen commented on HBASE-17782:
-
bq. Yechao Chen I guess you changed the description by accident and have revert
it, please let me know if any concern here. Thanks.
sorry for that,kid does:D,please ignore it.
> Extend IdReadWriteLock to support using both weak and soft reference
>
>
> Key: HBASE-17782
> URL: https://issues.apache.org/jira/browse/HBASE-17782
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17782.patch, HBASE-17782.patch
>
>
> As per discussed in HBASE-17747, we need to make it configurable to decide
> whether to use weak or soft reference for {{IdReadWriteLock}}, with soft
> reference by default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174755#comment-16174755
]
Abhishek Singh Chouhan commented on HBASE-18796:
Spent some time looking at the failure. Looks to be a problem elsewhere that
surfaced.
The test does a split and then tries a batch get operation which fails due to
table not found although the table is there. This is happening because now that
we do not put daughter locations before they're actually opened on the
regionserver, we run into NoServerForRegionException in
ConnectionImplementation#locateRegionInMeta which should be fine since there
are retries which should succeed as soon as the region is opened. However our
retry fails on a TableNotFound exception here
{code}
try (ReversedClientScanner rcs =
new ReversedClientScanner(conf, s, TableName.META_TABLE_NAME, this,
rpcCallerFactory,
rpcControllerFactory, getMetaLookupPool(),
metaReplicaCallTimeoutScanInMicroSecond)) {
regionInfoRow = rcs.next();
}
if (regionInfoRow == null) {
throw new TableNotFoundException(tableName);
}
{code}
The result that we get has mayHaveMoreCellsInRow() true during one of the
retries, since we don't have setAllowPartialResults(true) set on our scan we
get regionInfoRow as null since we got only 1 row which has
mayHaveMoreCellsInRow() as true and we use
CompleteScanResultCache which won't return this to the client. After i do
{code}
s.addFamily(HConstants.CATALOG_FAMILY);
s.setOneRowLimit();
+ s.setAllowPartialResults(true);
if (this.useMetaReplicas) {
s.setConsistency(Consistency.TIMELINE);
}
{code}
the client is able to ride over the split during its retries and the test
passes.
[~tedyu] [~apurtell] This issues seems to be something that can be hit during
any other retry too in locateRegionInMeta when mayHaveMoreCellsInRow() is true
for the meta scan and the client would get TableNotFound and will not retry. I
can open another jira for this if this sounds good.
> Admin#isTableAvailable returns incorrect result before daughter regions are
> opened
> --
>
> Key: HBASE-18796
> URL: https://issues.apache.org/jira/browse/HBASE-18796
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0
>
> Attachments: HBASE-18796.branch-1.001.patch,
> HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch,
> HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch
>
>
> Admin#isTableAvailable checks if it can getServerName for the meta entries it
> reads. During the time of split server location are added to the meta entries
> in MetaTableAccessor#splitRegion although the description of the method says
> "Does not add the location information to the daughter regions since they are
> not open yet.". At this point during the split daughter regions are not
> actually open, so we can get to a state where parent is offline, daughters
> are not yet open but isTableAvailable returns true.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174783#comment-16174783
]
Hadoop QA commented on HBASE-13346:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red}499m
54s{color} | {color:red} Docker failed to build yetus/hbase:5d60123. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-13346 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888211/HBASE-13346.master.003.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8722/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18856) take find security bugs for a spin
Sean Busbey created HBASE-18856: --- Summary: take find security bugs for a spin Key: HBASE-18856 URL: https://issues.apache.org/jira/browse/HBASE-18856 Project: HBase Issue Type: Improvement Components: security, test Reporter: Sean Busbey Priority: Minor it'd be nice to see if the [Find Security Bugs|http://find-sec-bugs.github.io/] plugin for FindBugs works now that we've switched to SpotBugs. Presuming it does, getting an initial include/exclude list going and things merged into our build could be a good addition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18731) [compat 1-2] QuotaSettings#setupSetQuotaRequest param changed
[
https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157740#comment-16157740
]
Sean Busbey edited comment on HBASE-18731 at 9/21/17 2:45 PM:
--
presumably this can also cover
{code}public static SetQuotaRequest buildSetQuotaRequestProto(final
QuotaSettings settings){code} in QuotaSettings? Or is there a different jira
for that already?
QuotaSettings is an Admin API and this is an instance of protobufs in our
public API. Better maybe to just release note it and make sure only a POJO is
within the public API for 2.0?
was (Author: busbey):
presumably this can also cover
{code}public static SetQuotaRequest buildSetQuotaRequestProto(final
QuotaSettings settings){code} in QuotaSettings? Or is there a different jira
for that already?
QuotaSettings is an Admin API and this is an instance of protobufs in our
public API. Better maybe to just release not it and make sure only a POJO is
within the public API for 2.0?
> [compat 1-2] QuotaSettings#setupSetQuotaRequest param changed
> -
>
> Key: HBASE-18731
> URL: https://issues.apache.org/jira/browse/HBASE-18731
> Project: HBase
> Issue Type: Sub-task
> Components: API
>Reporter: stack
>Assignee: Sean Busbey
> Fix For: 2.0.0-alpha-4
>
>
> QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to
> package
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest.
> QuotaSettings was added in 1.1.
> Is QuotaSettings for use by anyone but our CPEP?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18857) Update News section to point to video / slides from prior conferences
Sean Busbey created HBASE-18857: --- Summary: Update News section to point to video / slides from prior conferences Key: HBASE-18857 URL: https://issues.apache.org/jira/browse/HBASE-18857 Project: HBase Issue Type: Improvement Components: website Reporter: Sean Busbey or maybe we need a history of "talks / meetups" section. In either case, right now the new section is the closest we have to pointers on community provided resources. Adding in videos / slides would be a great addition to hte CFP pages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Summary: [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private (was: [compat 1-2] QuotaSettings#setupSetQuotaRequest param changed) > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Attachment: HBASE-18731.0.patch -0 master /branch-2 - mark the relevant methods as IA.Private for branch-1 and earlier, I'll add a Deprecated marker with a warning about the removal in 2+ > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Status: Patch Available (was: Open) > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174948#comment-16174948 ] Ted Yu commented on HBASE-18796: Since the issue was found soon after the commit, I would think applying addendum should suffice. Thanks for digging. > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174951#comment-16174951 ] ramkrishna.s.vasudevan commented on HBASE-16769: Comment on withCpCall already been said by [~chia7712]. That needs to be fixed. The CP hook is changing now. So since we are changing the signatures already it is fine. So we should revisit all CP hooks if there are any other similar hooks with unwanted hooks that is passing some private things and some unnecessary/additional things as param. Why is that we are newly passing the new param 'withCphook' and the other deleteSnapshot() API do not have that switch? Rest looks good to me. bq.AC should be internal, not as a CP? Good idea but probably a big change. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174998#comment-16174998 ] Ted Yu commented on HBASE-18651: Planning to commit later today if there is no further review comment. > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175001#comment-16175001 ] Reid Chan commented on HBASE-18651: --- ping [~mdrob], would you mind taking a look. > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[
https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175010#comment-16175010
]
Hadoop QA commented on HBASE-18731:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18731 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888313/HBASE-18731.0.patch |
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7ac0b8b2e269 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a6c3c64 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8723/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8723/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
[jira] [Commented] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175043#comment-16175043 ] stack commented on HBASE-17730: --- Yes is answer to your question above [~anoop.hbase] -- smile. I have list of incompatibilities for coprocessors running here https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.9k7mjbauv0wj but this issue is about more than just list of breakage; it is about adding section to 2.0 migration on how to move coprocessors over. > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17730: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0 > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17730: -- Component/s: documentation > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14845) hbase-server leaks jdk.tools dependency to mapreduce consumers
[
https://issues.apache.org/jira/browse/HBASE-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175048#comment-16175048
]
Sean Busbey commented on HBASE-14845:
-
this got fixed in HBase 2+ via the breakout into a mapreduce module:
{code}
[INFO]
[INFO] Building Apache HBase - MapReduce 2.0.0-alpha3-SNAPSHOT
[INFO]
[WARNING] The POM for
org.apache.hbase:hbase-server:jar:2.0.0-alpha3-20170913.213312-2 is invalid,
transitive dependencies (if any) will not be available, enable debug logging
for more details
[WARNING] The POM for
org.apache.hbase:hbase-server:jar:tests:2.0.0-alpha3-20170913.213312-2 is
invalid, transitive dependencies (if any) will not be available, enable debug
logging for more details
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-mapreduce
---
[INFO] org.apache.hbase:hbase-mapreduce:jar:2.0.0-alpha3-SNAPSHOT
[INFO] +-
org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:1.0.1:compile
[INFO] +- org.apache.hbase.thirdparty:hbase-shaded-netty:jar:1.0.1:compile
[INFO] +- org.apache.hbase.thirdparty:hbase-shaded-protobuf:jar:1.0.1:compile
[INFO] +- org.apache.hbase:hbase-common:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] | +- commons-codec:commons-codec:jar:1.10:compile
[INFO] | +- org.apache.commons:commons-collections4:jar:4.1:compile
[INFO] | \- org.apache.commons:commons-crypto:jar:1.0.0:compile
[INFO] +- org.apache.hbase:hbase-protocol:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] +-
org.apache.hbase:hbase-protocol-shaded:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-metrics:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-metrics-api:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- io.dropwizard.metrics:metrics-core:jar:3.2.1:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.7.24:compile
[INFO] +- org.apache.hbase:hbase-prefix-tree:jar:2.0.0-alpha3-SNAPSHOT:runtime
[INFO] +- org.apache.htrace:htrace-core:jar:3.2.0-incubating:compile
[INFO] +- org.apache.hbase:hbase-client:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] | +- org.jruby.jcodings:jcodings:jar:1.0.18:compile
[INFO] | +- org.jruby.joni:joni:jar:2.1.11:compile
[INFO] | +- org.apache.curator:curator-framework:jar:2.12.0:compile
[INFO] | +- org.apache.curator:curator-client:jar:2.12.0:compile
[INFO] | \- org.apache.hadoop:hadoop-auth:jar:2.7.1:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.3:compile
[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.6:compile
[INFO] | \-
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] |+-
org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] |+- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] |\- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] +- org.apache.hbase:hbase-hadoop-compat:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +-
org.apache.hbase:hbase-hadoop2-compat:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-server:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-replication:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- log4j:log4j:jar:1.2.17:compile
[INFO] +- commons-cli:commons-cli:jar:1.4:compile
[INFO] +- commons-io:commons-io:jar:2.5:compile
[INFO] +- org.apache.commons:commons-lang3:jar:3.6:compile
[INFO] +- commons-logging:commons-logging:jar:1.2:compile
[INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.10:compile
[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] +- com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
(optional)
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
[INFO] | +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | +- commons-net:commons-net:jar:3.1:compile
[INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] | +- org.apache.curator:curator-recipes:jar:2.12.0:compile
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | \- org.tukaani:xz:jar:1.0:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.1:compile
[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.1:compile
[INFO] | \- org.apache.hadoop:ha
[jira] [Resolved] (HBASE-14845) hbase-server leaks jdk.tools dependency to mapreduce consumers
[ https://issues.apache.org/jira/browse/HBASE-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-14845. --- Resolution: Fixed Fix Version/s: (was: 1.5.0) Good by me [~busbey] For now resolving as fixed in branch-2 by HBASE-18640 Move mapreduce out of hbase-server into separate module. Can reopen or make a new issue given it will be a different sort of solution fixing in branch-1. > hbase-server leaks jdk.tools dependency to mapreduce consumers > -- > > Key: HBASE-14845 > URL: https://issues.apache.org/jira/browse/HBASE-14845 > Project: HBase > Issue Type: Bug > Components: build, dependencies >Affects Versions: 2.0.0, 0.98.14, 1.2.0, 1.1.2, 1.3.0, 1.0.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-14845.1.patch > > > HBASE-13963 / HBASE-14844 take care of removing leaks of our dependency on > jdk-tools. > Until we move the mapreduce support classes out of hbase-server > (HBASE-11843), we need to also avoid leaking the dependency from that module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18105) [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world
[ https://issues.apache.org/jira/browse/HBASE-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175081#comment-16175081 ] Yi Liang commented on HBASE-18105: -- Hi [~stack], I need to a few clean up on hbase shell about split/merge, and also I saw the PONR has already removed in AMV2, but it still use PONR this name, maybe we need to rename the method, that will be enough. I am also trying to write some test cases about the PONR to see if it really works fine. > [AMv2] Split/Merge need cleanup; currently they diverge and do not fully > embrace AMv2 world > --- > > Key: HBASE-18105 > URL: https://issues.apache.org/jira/browse/HBASE-18105 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack > Fix For: 2.0.0 > > > Region Split and Merge work on the new AMv2 but they work differently. This > issue is about bringing them back together and fully embracing the AMv2 > program. > They both have issues mostly the fact that they carry around baggage no > longer necessary in the new world of assignment. > Here are some of the items: > Split and Merge metadata modifications are done by the Master now but we have > vestige of Split/Merge on RS still; e.g. when we SPLIT, we ask the Master > which asks the RS, which turns around, and asks the Master to run the > operation. Fun. MERGE is all done Master-side. > > Clean this up. Remove asking RS to run SPLIT and remove RegionMergeRequest, > etc. on RS-side. Also remove PONR. We don’t Points-Of-No-Return now we are up > on Pv2. Remove all calls in Interfaces; they are unused. Make RS still able > to detect when split, but have it be a client of Master like anyone else. > Split is Async but does not return procId > Split is async. Doesn’t return the procId though. Merge does. Fix. Only hard > part here I think is the Admin API does not allow procid return. > Flags > Currently OFFLINE is determined by looking either at the master instance of > HTD (isOffline) and/or at the RegionState#state. Ditto for SPLIT. For MERGE, > we rely on RegionState#state. Related is a note above on how split works -- > there is a split flag in HTD when there should not be. > > TODO is move to rely on RegionState#state exclusively in Master. > From Split/Merge Procedures need finishing in > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.4b60dc1h4m1f -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175084#comment-16175084 ] stack commented on HBASE-18731: --- +1 on patch for branch-2 > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18105) [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world
[ https://issues.apache.org/jira/browse/HBASE-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175089#comment-16175089 ] stack commented on HBASE-18105: --- Yeah, the PONR qualification needs to go in AMv2. I love it that you are writing tests. Thank you. > [AMv2] Split/Merge need cleanup; currently they diverge and do not fully > embrace AMv2 world > --- > > Key: HBASE-18105 > URL: https://issues.apache.org/jira/browse/HBASE-18105 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack > Fix For: 2.0.0 > > > Region Split and Merge work on the new AMv2 but they work differently. This > issue is about bringing them back together and fully embracing the AMv2 > program. > They both have issues mostly the fact that they carry around baggage no > longer necessary in the new world of assignment. > Here are some of the items: > Split and Merge metadata modifications are done by the Master now but we have > vestige of Split/Merge on RS still; e.g. when we SPLIT, we ask the Master > which asks the RS, which turns around, and asks the Master to run the > operation. Fun. MERGE is all done Master-side. > > Clean this up. Remove asking RS to run SPLIT and remove RegionMergeRequest, > etc. on RS-side. Also remove PONR. We don’t Points-Of-No-Return now we are up > on Pv2. Remove all calls in Interfaces; they are unused. Make RS still able > to detect when split, but have it be a client of Master like anyone else. > Split is Async but does not return procId > Split is async. Doesn’t return the procId though. Merge does. Fix. Only hard > part here I think is the Admin API does not allow procid return. > Flags > Currently OFFLINE is determined by looking either at the master instance of > HTD (isOffline) and/or at the RegionState#state. Ditto for SPLIT. For MERGE, > we rely on RegionState#state. Related is a note above on how split works -- > there is a split flag in HTD when there should not be. > > TODO is move to rely on RegionState#state exclusively in Master. > From Split/Merge Procedures need finishing in > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.4b60dc1h4m1f -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175109#comment-16175109 ] stack commented on HBASE-16769: --- [~anoop.hbase] Thanks for pointing me at the original JIRA that added this hook. Is this call any good w/o the Cells? The AC won't know what you are trying to replicate so can only say yes or no, nothing more fine-grained that this. Can we do an awful hack that marks it Private for AC only -- DO NOT USE -- until we move AC off CPs and internal? > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175105#comment-16175105 ] Anoop Sam John commented on HBASE-16769: This method of listing snapshot is used from another method which is just to verify whether snapshots are supported. There no need to call CPs.. So the extra param is added. Ya I have fixed comment on that boolean. Will commit that only.. Can I get some +1s? > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175121#comment-16175121 ] Anoop Sam John commented on HBASE-16769: That time the hook was added as general purpose thing.. But ya may be not really needed.. Ya for AC there is no need to see the cells which are replicated. AC wants to know the user and his permission levels only. I can add a private annotation at the method (and post also) and add some comments that this should not be used at all. Or may be we can even deprecate that also. No need to add private annotate. Can say this is in place just to support our AC and this might get removed at any time after 2.0. May be we will end up in min 3.0 release for such a removal (Depends on AC rework).. So better we can say deprecated from 2.0 with out any replacement. Wont mention will remove in 3.0 as we are not sure :-) Deprecated with out any replacement itself will warn users from using this > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-18817: --- Assignee: Sean Busbey > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18758) [compat 1-2] Test delegation tokens continue to work when hbase1 going against hbase2 cluster
[ https://issues.apache.org/jira/browse/HBASE-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18758: Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > [compat 1-2] Test delegation tokens continue to work when hbase1 going > against hbase2 cluster > - > > Key: HBASE-18758 > URL: https://issues.apache.org/jira/browse/HBASE-18758 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Priority: Critical > Fix For: 2.0.0-beta-1 > > > Suggested by [~toffer] Need to test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15666) shaded dependencies for hbase-testing-util
[ https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15666: Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > shaded dependencies for hbase-testing-util > -- > > Key: HBASE-15666 > URL: https://issues.apache.org/jira/browse/HBASE-15666 > Project: HBase > Issue Type: New Feature > Components: test >Affects Versions: 1.1.0, 1.2.0 >Reporter: Sean Busbey >Priority: Critical > Fix For: 1.5.0, 2.0.0-beta-1 > > > Folks that make use of our shaded client but then want to test things using > the hbase-testing-util end up getting all of our dependencies again in the > test scope. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18786) FileNotFoundException should not be silently handled for primary region replicas
[ https://issues.apache.org/jira/browse/HBASE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175160#comment-16175160 ] Andrew Purtell commented on HBASE-18786: Precommit on master is still messed up. I'm going to commit based on previous +1 and comments later today unless objection. Let me run the full suite for branch-2 and branch-1 with this change in place first and make sure any failures still fail without this patch in place. > FileNotFoundException should not be silently handled for primary region > replicas > > > Key: HBASE-18786 > URL: https://issues.apache.org/jira/browse/HBASE-18786 > Project: HBase > Issue Type: Sub-task > Components: regionserver, Scanners >Reporter: Ashu Pachauri >Assignee: Andrew Purtell > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18786-branch-1.patch, HBASE-18786-branch-1.patch, > HBASE-18786.patch, HBASE-18786.patch > > > This is a follow up for HBASE-18186. > FileNotFoundException while scanning from a primary region replica can be > indicative of a more severe problem. Handling them silently can cause many > underlying issues go undetected. We should either > 1. Hard fail the regionserver if there is a FNFE on a primary region replica, > OR > 2. Report these exceptions as some region / server level metric so that these > can be proactively investigated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175167#comment-16175167 ] stack commented on HBASE-16769: --- bq. That time the hook was added as general purpose thing.. But ya may be not really needed.. Ya for AC there is no need to see the cells which are replicated. AC wants to know the user and his permission levels only. Ok. So, we think this method is going to go away in 3.0... hopefully because AC gets off CP and becomes integral? If so, we should try not to keep people off it. The bad thing though is that if the method is in the Interface, they'll have to implement it, right? Unless we do the @appy refactor first and put in place defaults? If we can have jdk8 defaults, then I like your idea of deprecate+private+don't use because for AC which is going internal > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[
https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175177#comment-16175177
]
Anoop Sam John commented on HBASE-16769:
Yes we do have default impl in the interface
default void preReplicateLogEntries(final
ObserverContext ctx)
+ throws IOException {
+ }
So we are goo with it.
> Deprecate/remove PB references from MasterObserver and RegionServerObserver
> ---
>
> Key: HBASE-16769
> URL: https://issues.apache.org/jira/browse/HBASE-16769
> Project: HBase
> Issue Type: Bug
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-16769.patch, HBASE-16769_V2.patch
>
>
> This is effectively a sub-task for HBASE-15174.
> CP Methods
> MasterObserver
> preListSnapshot
> postListSnapshot
> preSnapshot
> postSnapshot
> preCloneSnapshot
> postCloneSnapshot
> preRestoreSnapshot
> postRestoreSnapshot
> preDeleteSnapshot
> postDeleteSnapshot
>
> preSetUserQuota
> postSetUserQuota
> preSetUserQuota
> postSetUserQuota
> preSetUserQuota
> postSetUserQuota
> preSetTableQuota
> postSetTableQuota
> preSetNamespaceQuota
> postSetNamespaceQuota
>
> RegionServerObserver
> preReplicateLogEntries
> postReplicateLogEntries
> Note : This issue not handling Quota related CPs. Same is handled by a
> subtask here HBase-18807
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[ https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175184#comment-16175184 ] Vladimir Rodionov edited comment on HBASE-18843 at 9/21/17 5:44 PM: Patch v1 cc: [~te...@apache.org] was (Author: vrodionov): Patch v1 > Add DistCp support to incremental backup with bulk loading > -- > > Key: HBASE-18843 > URL: https://issues.apache.org/jira/browse/HBASE-18843 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18843-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175185#comment-16175185 ] stack commented on HBASE-16769: --- ok. so, you need to add to the patch "DON'T USE"?... > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[ https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-18843: -- Attachment: HBASE-18843-v1.patch Patch v1 > Add DistCp support to incremental backup with bulk loading > -- > > Key: HBASE-18843 > URL: https://issues.apache.org/jira/browse/HBASE-18843 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18843-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[ https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-18843: -- Description: Currently, we copy bulk loaded files to backup one-by-one on a client side (where backup create runs). This has to be replaced with DistCp copying. > Add DistCp support to incremental backup with bulk loading > -- > > Key: HBASE-18843 > URL: https://issues.apache.org/jira/browse/HBASE-18843 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18843-v1.patch > > > Currently, we copy bulk loaded files to backup one-by-one on a client side > (where backup create runs). This has to be replaced with DistCp copying. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18787) Fix the "dependencies connecting to an HBase cluster"
[
https://issues.apache.org/jira/browse/HBASE-18787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175191#comment-16175191
]
stack commented on HBASE-18787:
---
So, how to proceed here? Gate this on a section being added on hbase shaded
client so we can reference it? Or just commit this and then fix it up when
shaded client section goes in? The latter is fine by me [~chia7712]
> Fix the "dependencies connecting to an HBase cluster"
> -
>
> Key: HBASE-18787
> URL: https://issues.apache.org/jira/browse/HBASE-18787
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18787.v0.patch
>
>
> {code}
> Minimally, an HBase client needs several libraries in its CLASSPATH when
> connecting to a cluster, including:
> commons-configuration (commons-configuration-1.6.jar)
> commons-lang3 (commons-lang3-3.6.jar)
> commons-logging (commons-logging-1.1.1.jar)
> hadoop-core (hadoop-core-1.0.0.jar)
> hbase (hbase-0.92.0.jar)
> log4j (log4j-1.2.16.jar)
> slf4j-api (slf4j-api-1.5.8.jar)
> slf4j-log4j (slf4j-log4j12-1.5.8.jar)
> zookeeper (zookeeper-3.4.2.jar)
> {code}
> The libraries listed in our docs are stale. Also, it may be better to say
> "*Minimally, an HBase client needs hbase-client module in its dependencies
> when connecting to a cluster*" rather than "*Minimally, an HBase client needs
> several libraries in its CLASSPATH when connecting to a cluster,
> including:*". I don't think newbie want to add all dependencies manually.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175192#comment-16175192 ] Anoop Sam John commented on HBASE-16769: I prefer for this pre and post hooks 1. Add deprecation from 2.0 with out any replacement 2. Say DON'T USE and this is maintained just for internal use. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[
https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175194#comment-16175194
]
Ted Yu commented on HBASE-18843:
{code}
+public class FixedRelativePathCopyListing extends SimpleCopyListing {
{code}
Add annotation for audience.
Please put patch on review board.
{code}
+ private SequenceFile.Writer getWriter(Path pathToListFile) throws
IOException {
{code}
Have you considered using other format which is more amiable to upgrade ?
Can you add some test for this new class ?
> Add DistCp support to incremental backup with bulk loading
> --
>
> Key: HBASE-18843
> URL: https://issues.apache.org/jira/browse/HBASE-18843
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18843-v1.patch
>
>
> Currently, we copy bulk loaded files to backup one-by-one on a client side
> (where backup create runs). This has to be replaced with DistCp copying.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-15071) Cleanup bypass semantic in MasterCoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-15071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe reassigned HBASE-15071: Assignee: Umesh Agashe > Cleanup bypass semantic in MasterCoprocessorHost > > > Key: HBASE-15071 > URL: https://issues.apache.org/jira/browse/HBASE-15071 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 2.0.0 >Reporter: stack >Assignee: Umesh Agashe >Priority: Blocker > Attachments: HBASE-15071.patch > > > Lets decide on this one before we release 2.0.0. > A bunch of methods in MasterCoprocessorHost on the 'pre' step allow returning > true which indicates the method invocation is not to proceed. > Not all 'pre' steps do this. Just some. > Seems a little arbitrary. > How we skip out if we are not proceed with the invocation is also a little > arbitrary. > When a deleteColumn call is supposed to skip out, it returns a -1, a > non-procId. If we are to skip a balance call, we log that CP said skip and > then return false to indicate the balancer did not run (why?). Elsewhere we > just exit silently. In createNamespace we used to exit silently but > HBASE-14888 just changed it so we throw a BypassCoprocessorException > instead... > Lets make them all work the same way. > (This issue comes of chat w/ Matteo) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[
https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175216#comment-16175216
]
Ted Yu commented on HBASE-18843:
In this method:
{code}
+ public void doBuildListing(SequenceFile.Writer fileListWriter,
+ DistCpOptions options) throws IOException {
{code}
There is quite some duplicate code with the SimpleCopyListing class.
Can you do some refactoring ?
> Add DistCp support to incremental backup with bulk loading
> --
>
> Key: HBASE-18843
> URL: https://issues.apache.org/jira/browse/HBASE-18843
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18843-v1.patch
>
>
> Currently, we copy bulk loaded files to backup one-by-one on a client side
> (where backup create runs). This has to be replaced with DistCp copying.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17399) region_mover.rb uses different default filename, also needs slight change
[
https://issues.apache.org/jira/browse/HBASE-17399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175228#comment-16175228
]
ChunHao commented on HBASE-17399:
-
[~stack] An original example for a generated file is:
/tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
The result in a rather strange \:. Therefore, I deleted the '\'.
Besides, To make the result easy to handle, I added a divider '#' between the
username and the hostname, and I also added a divider '#' between the hostname
and the port.
Thus, the original example becomes
"/tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020"
Finally, I corrected the command line help message:
the original help message was "default /tmp/", now the help
message is "default /tmp/".
> region_mover.rb uses different default filename, also needs slight change
> -
>
> Key: HBASE-17399
> URL: https://issues.apache.org/jira/browse/HBASE-17399
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: ChunHao
> Labels: beginner
> Attachments: HBASE-17399.branch-1.3.v0.patch,
> HBASE-17399.branch-1.3.v1.patch
>
>
> The command line help prints:
> {noformat}
> -f, --filename=FILE File to save regions list into unloading, \
> or read from loading; default /tmp/
> {noformat}
> while in reality, the code does this:
> {code}
> def getFilename(options, targetServer, port)
> filename = options[:file]
> if not filename
> filename = "/tmp/" + ENV['USER'] + targetServer + ":" + port
> end
> return filename
> end
> {code}
> An example for a generated file is:
> {noformat}
> /tmp/larsgeorgeslave-3.internal.larsgeorge.com\:16020
> {noformat}
> I suggest we fix the command line help explanation. But first, we should also
> fix how the name is generated, *adding* a divider between the user name and
> the host name, and also change how the port is attached to the host name.
> Currently this results in a rather strange {{\:}} which could be hard to
> handle. Maybe we simply use an exclamation mark or hash for both? For example:
> {noformat}
> /tmp/larsgeorge!slave-3.internal.larsgeorge.com!16020
> /tmp/larsgeorge#slave-3.internal.larsgeorge.com#16020
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18787) Fix the "dependencies connecting to an HBase cluster"
[
https://issues.apache.org/jira/browse/HBASE-18787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175239#comment-16175239
]
Chia-Ping Tsai commented on HBASE-18787:
bq. Or just commit this and then fix it up when shaded client section goes in?
The latter is fine by me Chia-Ping Tsai
We are on the same page. :)
> Fix the "dependencies connecting to an HBase cluster"
> -
>
> Key: HBASE-18787
> URL: https://issues.apache.org/jira/browse/HBASE-18787
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18787.v0.patch
>
>
> {code}
> Minimally, an HBase client needs several libraries in its CLASSPATH when
> connecting to a cluster, including:
> commons-configuration (commons-configuration-1.6.jar)
> commons-lang3 (commons-lang3-3.6.jar)
> commons-logging (commons-logging-1.1.1.jar)
> hadoop-core (hadoop-core-1.0.0.jar)
> hbase (hbase-0.92.0.jar)
> log4j (log4j-1.2.16.jar)
> slf4j-api (slf4j-api-1.5.8.jar)
> slf4j-log4j (slf4j-log4j12-1.5.8.jar)
> zookeeper (zookeeper-3.4.2.jar)
> {code}
> The libraries listed in our docs are stale. Also, it may be better to say
> "*Minimally, an HBase client needs hbase-client module in its dependencies
> when connecting to a cluster*" rather than "*Minimally, an HBase client needs
> several libraries in its CLASSPATH when connecting to a cluster,
> including:*". I don't think newbie want to add all dependencies manually.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18800) [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going forward
[ https://issues.apache.org/jira/browse/HBASE-18800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175251#comment-16175251 ] stack commented on HBASE-18800: --- This needs doc'ing in refguide. > [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going > forward > -- > > Key: HBASE-18800 > URL: https://issues.apache.org/jira/browse/HBASE-18800 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 2.0.0-beta-1 > > > We've bandied this about for a while now that folks should consume hbase via > the shaded hbase-client; it should work if their needs are minimal (and if it > doesn't work, would be good to hear why). This issue is about evangelizing > the shaded hbase-client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18858) Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
Chia-Ping Tsai created HBASE-18858:
--
Summary: Avoid NPE occurring in
TestReplicationBase#tearDownAfterClass
Key: HBASE-18858
URL: https://issues.apache.org/jira/browse/HBASE-18858
Project: HBase
Issue Type: Bug
Components: test
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Minor
Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1
{code}
public static void tearDownAfterClass() throws Exception {
htable2.close();
htable1.close();
admin.close();
utility2.shutdownMiniCluster();
utility1.shutdownMiniCluster();
}
{code}
If the setUpBeforeClass() fail, some members will be NULL. We need to ensures
all resources are closed at the end of tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175264#comment-16175264 ] stack commented on HBASE-16769: --- Sounds good to me [~anoop.hbase] Put up patch so I can +1 it. Add that the internal use is by AC for us devs reading along afterward. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175269#comment-16175269
]
stack commented on HBASE-13346:
---
I just read HBASE-18797 My review was done BEFORE I read HBASE-18797 (In other
words, I don't think it very useful).
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18838) shaded artifacts are incorrect when built against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18838: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > shaded artifacts are incorrect when built against hadoop 3 > -- > > Key: HBASE-18838 > URL: https://issues.apache.org/jira/browse/HBASE-18838 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0-beta-1 > > > Building master/branch-2 against the hadoop-3 profile results in > check-invariants screaming about unrelocated dependencies. will list details > in comment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18858) Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
[
https://issues.apache.org/jira/browse/HBASE-18858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18858:
---
Status: Patch Available (was: Open)
> Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
> -
>
> Key: HBASE-18858
> URL: https://issues.apache.org/jira/browse/HBASE-18858
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1
>
> Attachments: HBASE-18858.branch-1.v0.patch
>
>
> {code}
> public static void tearDownAfterClass() throws Exception {
> htable2.close();
> htable1.close();
> admin.close();
> utility2.shutdownMiniCluster();
> utility1.shutdownMiniCluster();
> }
> {code}
> If the setUpBeforeClass() fail, some members will be NULL. We need to ensures
> all resources are closed at the end of tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18858) Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
[
https://issues.apache.org/jira/browse/HBASE-18858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18858:
---
Attachment: HBASE-18858.branch-1.v0.patch
> Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
> -
>
> Key: HBASE-18858
> URL: https://issues.apache.org/jira/browse/HBASE-18858
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1
>
> Attachments: HBASE-18858.branch-1.v0.patch
>
>
> {code}
> public static void tearDownAfterClass() throws Exception {
> htable2.close();
> htable1.close();
> admin.close();
> utility2.shutdownMiniCluster();
> utility1.shutdownMiniCluster();
> }
> {code}
> If the setUpBeforeClass() fail, some members will be NULL. We need to ensures
> all resources are closed at the end of tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17852: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental > backup) > > > Key: HBASE-17852 > URL: https://issues.apache.org/jira/browse/HBASE-17852 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-17852-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work stopped] (HBASE-18838) shaded artifacts are incorrect when built against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18838 stopped by Sean Busbey. --- > shaded artifacts are incorrect when built against hadoop 3 > -- > > Key: HBASE-18838 > URL: https://issues.apache.org/jira/browse/HBASE-18838 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0-beta-1 > > > Building master/branch-2 against the hadoop-3 profile results in > check-invariants screaming about unrelocated dependencies. will list details > in comment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175272#comment-16175272 ] stack commented on HBASE-17852: --- Moving out of alpha-4. If it comes in before alpha-4, that'd be great but no critical to the coprocessor-focused release. > Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental > backup) > > > Key: HBASE-17852 > URL: https://issues.apache.org/jira/browse/HBASE-17852 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-17852-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18838) shaded artifacts are incorrect when built against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-18838: --- Assignee: (was: Sean Busbey) I'll be happy to pick this back up when I start looking at beta-1 stuff, but I'm setting it back to unassigned at the moment in case someone else wants to dig into it earlier. > shaded artifacts are incorrect when built against hadoop 3 > -- > > Key: HBASE-18838 > URL: https://issues.apache.org/jira/browse/HBASE-18838 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Priority: Critical > Fix For: 2.0.0-beta-1 > > > Building master/branch-2 against the hadoop-3 profile results in > check-invariants screaming about unrelocated dependencies. will list details > in comment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Fix Version/s: 3.0.0 > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Attachment: HBASE-18731-branch-1.v1.patch -v1 branch-1 - mark QuotaSettings.buildSetQuotaRequestProto and QuotaSettings.setupSetQuotaRequest as deprecated with a note that they're gone in 2.0+ > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175300#comment-16175300 ] Sean Busbey commented on HBASE-18731: - (I pushed the master/branch-2 version. thanks for the quick review on that!) > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18703) Inconsistent behavior for preBatchMutate in doMiniBatchMutate and processRowsWithLocks
[ https://issues.apache.org/jira/browse/HBASE-18703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175318#comment-16175318 ] Umesh Agashe edited comment on HBASE-18703 at 9/21/17 7:23 PM: --- [~anoop.hbase], [~Apache9]: These two code paths look very similar and I found comments/ todos in the code about unifying these two paths. The main difference I see here is: * one is atomic and other is not * Atomic path guarantees the oder of operations and non-atomic path does not To remove inconsistencies (above mentioned + a few other minor), I am working on a patch to unify these paths. I think I will be able to upload the patch soon. Let me know your thoughts. was (Author: uagashe): [~anoopamz], [~Apache9]: These two code paths look very similar and I found comments/ todos in the code about unifying these two paths. The main difference I see here is: * one is atomic and other is not * Atomic path guarantees the oder of operations and non-atomic path does not To remove inconsistencies (above mentioned + a few other minor), I am working on a patch to unify these paths. I think I will be able to upload the patch soon. Let me know your thoughts. > Inconsistent behavior for preBatchMutate in doMiniBatchMutate and > processRowsWithLocks > -- > > Key: HBASE-18703 > URL: https://issues.apache.org/jira/browse/HBASE-18703 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Umesh Agashe >Priority: Critical > Fix For: 2.0.0-alpha-4 > > > In doMiniBatchMutate, the preBatchMutate is called before building WAL, but > in processRowsWithLocks, we suggest the RowProcessor implementation to build > WAL in process method, which is ahead of preBatchMutate. > If a CP modifies the mutations, especially if it removes some cells from the > mutations, then the behavior of processRowsWithLocks is broken. The changes > applied to memstore and WAL will be different. And there is no way to remove > entries from a WALEdit through CP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18703) Inconsistent behavior for preBatchMutate in doMiniBatchMutate and processRowsWithLocks
[ https://issues.apache.org/jira/browse/HBASE-18703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175318#comment-16175318 ] Umesh Agashe commented on HBASE-18703: -- [~anoopamz], [~Apache9]: These two code paths look very similar and I found comments/ todos in the code about unifying these two paths. The main difference I see here is: * one is atomic and other is not * Atomic path guarantees the oder of operations and non-atomic path does not To remove inconsistencies (above mentioned + a few other minor), I am working on a patch to unify these paths. I think I will be able to upload the patch soon. Let me know your thoughts. > Inconsistent behavior for preBatchMutate in doMiniBatchMutate and > processRowsWithLocks > -- > > Key: HBASE-18703 > URL: https://issues.apache.org/jira/browse/HBASE-18703 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Umesh Agashe >Priority: Critical > Fix For: 2.0.0-alpha-4 > > > In doMiniBatchMutate, the preBatchMutate is called before building WAL, but > in processRowsWithLocks, we suggest the RowProcessor implementation to build > WAL in process method, which is ahead of preBatchMutate. > If a CP modifies the mutations, especially if it removes some cells from the > mutations, then the behavior of processRowsWithLocks is broken. The changes > applied to memstore and WAL will be different. And there is no way to remove > entries from a WALEdit through CP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Singh Chouhan reopened HBASE-18796: Reopening to add addendum > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18817: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Singh Chouhan updated HBASE-18796: --- Attachment: HBASE-18796-addendum.master.patch Addendum for master that allows partial scan so as to not get incorrect TableNotFoundException > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796-addendum.master.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.002.patch, HBASE-18796.branch-1.003.patch, > HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18596) A hbase1 cluster should be able to replicate to a hbase2 cluster; verify
[ https://issues.apache.org/jira/browse/HBASE-18596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18596: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > A hbase1 cluster should be able to replicate to a hbase2 cluster; verify > > > Key: HBASE-18596 > URL: https://issues.apache.org/jira/browse/HBASE-18596 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: Esteban Gutierrez >Priority: Blocker > Fix For: 2.0.0-beta-1 > > > From the mailing list thread "[DISCUSS] hbase-2.0.0 compatibility > expectations", [~esteban] asks: > bq. Should we add additional details around replication as well? for > instance, shall we consider a hbase-1.x cluster as a client for a hbase-2.x > cluster? > The latter should be a blocker. Verify it works. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Singh Chouhan updated HBASE-18796: --- Status: Patch Available (was: Reopened) > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796-addendum.master.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.002.patch, HBASE-18796.branch-1.003.patch, > HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18817 started by Sean Busbey. --- > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Singh Chouhan updated HBASE-18796: --- Attachment: HBASE-18796-addendum.branch-1.patch Getting pre commit run for branch-1 > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796-addendum.branch-1.patch, > HBASE-18796-addendum.master.patch, HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18817: Attachment: HBASE-18817-branch-2.v0.patch -0 - removes hbase-spark module - removes hbase-spark-it module - removes spark chapter in refguide - removes hbase-spark* from assemblies > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18817-branch-2.v0.patch > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18817: Status: Patch Available (was: In Progress) > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18817-branch-2.v0.patch > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18859) Purge PB from BulkLoadObserver
stack created HBASE-18859: - Summary: Purge PB from BulkLoadObserver Key: HBASE-18859 URL: https://issues.apache.org/jira/browse/HBASE-18859 Project: HBase Issue Type: Sub-task Reporter: stack Noticed by [~appy] We expose PBs in this CP. Pass POJOs. This is like [~anoop.hbase] and [~elserj] pb purging that is going on contemporaneously. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[
https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175355#comment-16175355
]
Hadoop QA commented on HBASE-18731:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
56s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
0s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
21s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 28s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
7s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f1cc2c |
| JIRA Issue | HBASE-18731 |
| JIRA Patch URL |
https://issues.apache.o
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175419#comment-16175419
]
Hadoop QA commented on HBASE-18796:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
29s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18796 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888357/HBASE-18796-addendum.master.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 70f172fa95a2 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e393599 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8726/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8726/console |
| Powered by | Apache Yetus 0.4.0 http://yet
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175457#comment-16175457
]
Hadoop QA commented on HBASE-18796:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
46s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
39s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 40s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
3s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f1cc2c |
| JIRA Issue | HBASE-18796 |
| JIRA Patch URL |
https://issues.apache.o
[jira] [Commented] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175461#comment-16175461 ] Zach York commented on HBASE-18840: --- [~stack] Yes, I am developing against branch-1 (more specifically 1.3.1, but it is similar enough). In the near future, I expect I will start deving against branch-2. Since the other issue was resolved, I'll just put up a patch and you can look at the specific implementation. It is possible some of it doesn't make sense with newer features in branch-2, but we can discuss that there. > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18825) Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface
[ https://issues.apache.org/jira/browse/HBASE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175470#comment-16175470 ] stack commented on HBASE-18825: --- Ok. Thanks for the philosophy. That helps. So, we disabuse ourselves of any notion that another might want to put in place their own HRegion implemenation -- whether a subclass or a new implementation altogether -- if only because we've never done the work to ensure it is supported? I'm good w/ that (+1 on cleanup of "hbase.hregion.impl" that [~anoop.hbase] helpfully turned up). +1 too on using implementation internally rather than Interfaces. Going via Interfaces which are NOT IA.Private means we limit our ability to change. I like this observation. We need to put it up on the hbase dev billboard (I can add to dev section in guide at least). We make same call for entities below HRegion so, it is not possible to provide an alternate Store? I'm good with that. What about StoreFile? I'm thinking about alternate HFile implementations. What if someone wants to plug in a columnar-based file format? Or we want to do a V4 of HFile. This is harder for me to swallow. Looking at your patch though, I see that you do not disturb StoreFileReader nor StoreFileWriter. So it seems like alternate HFile implementations are still possible? If so, good. That Region and Store were made to feed CP is also an interesting observation. It seems like Region gets pretty wide usage in the codebase since its introduction, beyond CP-only use. No harm I suppose. So, I buy-in but need clarification around new HFile implementations. I'd also like to help message this philosophy so it sees adoption beyond [~Apache9] contribs. > Use HStoreFile instead of StoreFile in our own code base and remove > unnecessary methods in StoreFile interface > -- > > Key: HBASE-18825 > URL: https://issues.apache.org/jira/browse/HBASE-18825 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18825.patch, HBASE-18825-v1.patch, > HBASE-18825-v2.patch, HBASE-18825-v3.patch, HBASE-18825-v3.patch > > > Use generic types to avoid too many casts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18825) Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface
[ https://issues.apache.org/jira/browse/HBASE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175472#comment-16175472 ] stack commented on HBASE-18825: --- Oh, yeah, as suggested above, needs airing on dev list if only to bring folks attention to the tendency. > Use HStoreFile instead of StoreFile in our own code base and remove > unnecessary methods in StoreFile interface > -- > > Key: HBASE-18825 > URL: https://issues.apache.org/jira/browse/HBASE-18825 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18825.patch, HBASE-18825-v1.patch, > HBASE-18825-v2.patch, HBASE-18825-v3.patch, HBASE-18825-v3.patch > > > Use generic types to avoid too many casts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18807) Remove PB references from Observers for Quotas
[
https://issues.apache.org/jira/browse/HBASE-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-18807:
---
Attachment: HBASE-18807.003.branch-2.patch
.003 Lots of cleanup. Some missing test annotations. Marked
{{GlobalQuotaSettings}} as {{LimitedPrivate(COPROC)}}. Better javadoc.
> Remove PB references from Observers for Quotas
> --
>
> Key: HBASE-18807
> URL: https://issues.apache.org/jira/browse/HBASE-18807
> Project: HBase
> Issue Type: Sub-task
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18807.001.branch-2.patch,
> HBASE-18807.002.branch-2.patch, HBASE-18807.003.branch-2.patch
>
>
> Break-out from the parent:
> Same idea, just applied to the Observer methods for pre/post quota
> operations. Requires changes to MasterQuotaManager and the QuotaSettings
> implementations as some business logic is written on the PB objects directly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Attachment: HBASE-18840.HBASE-18477.002.patch > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Attachment: HBASE-18840.HBASE-18477.003.patch > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch, HBASE-18840.HBASE-18477.003.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Status: Patch Available (was: Open) > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch, HBASE-18840.HBASE-18477.003.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Affects Version/s: HBASE-18477 > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch, HBASE-18840.HBASE-18477.003.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rich Howarth updated HBASE-18847:
-
Attachment: HBASE-18847.v1.patch
> remove unneeded synchronized block from hfilev2 warning in branch-1.2
> -
>
> Key: HBASE-18847
> URL: https://issues.apache.org/jira/browse/HBASE-18847
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6
>Reporter: Rich Howarth
>Assignee: Rich Howarth
>Priority: Critical
> Fix For: 1.2.7
>
> Attachments: HBASE-18847.v1.patch
>
>
> The below code block starts at line 277 of HFileWriterV2.java. Class-level
> synchronization in a heavily used code path has a demonstrably significant
> negative effect on performance. I tested forcing a major compaction with 18
> compaction threads per node; removing the synchronization resulted in an
> order of magnitude performance increase, with the bottleneck then being at
> the disks (where I want it to be).
> {code:java}
> synchronized (HFileWriterV2.class) {
> if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) {
> LOG.warn("A minimum HFile version of " +
> HFile.MIN_FORMAT_VERSION_WITH_TAGS
> + " is required to support cell attributes/tags. Consider setting "
> + HFile.FORMAT_VERSION_KEY + " accordingly.");
> WARN_CELL_WITH_TAGS = false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175496#comment-16175496 ] Dave Latham commented on HBASE-12081: - Java 9 is now GA > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Attachment: HBASE-18840.HBASE-18477.001.patch > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rich Howarth updated HBASE-18847:
-
Status: Patch Available (was: Open)
> remove unneeded synchronized block from hfilev2 warning in branch-1.2
> -
>
> Key: HBASE-18847
> URL: https://issues.apache.org/jira/browse/HBASE-18847
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0
>Reporter: Rich Howarth
>Assignee: Rich Howarth
>Priority: Critical
> Fix For: 1.2.7
>
> Attachments: HBASE-18847.v1.patch
>
>
> The below code block starts at line 277 of HFileWriterV2.java. Class-level
> synchronization in a heavily used code path has a demonstrably significant
> negative effect on performance. I tested forcing a major compaction with 18
> compaction threads per node; removing the synchronization resulted in an
> order of magnitude performance increase, with the bottleneck then being at
> the disks (where I want it to be).
> {code:java}
> synchronized (HFileWriterV2.class) {
> if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) {
> LOG.warn("A minimum HFile version of " +
> HFile.MIN_FORMAT_VERSION_WITH_TAGS
> + " is required to support cell attributes/tags. Consider setting "
> + HFile.FORMAT_VERSION_KEY + " accordingly.");
> WARN_CELL_WITH_TAGS = false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175499#comment-16175499 ] stack commented on HBASE-12081: --- Making this a blocker for 2.0.0. Ensure we at least run on a jdk9. > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12081: -- Fix Version/s: 2.0.0 > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12081: -- Priority: Blocker (was: Major) > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175501#comment-16175501 ] Andrew Purtell commented on HBASE-18796: I'm running full test suites for HBASE-18786. I have rolled in the addendums as separate commits in those working branches so will test them both together. If good I will commit them. Thanks for the addendum [~abhishek.chouhan] and thanks for pointing out the issue [~ted_yu] > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796-addendum.branch-1.patch, > HBASE-18796-addendum.master.patch, HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[ https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18477 started by Zach York. - > Umbrella JIRA for HBase Read Replica clusters > - > > Key: HBASE-18477 > URL: https://issues.apache.org/jira/browse/HBASE-18477 > Project: HBase > Issue Type: New Feature >Reporter: Zach York >Assignee: Zach York > Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase > Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope > doc_v2.docx > > > Recently, changes (such as HBASE-17437) have unblocked HBase to run with a > root directory external to the cluster (such as in Amazon S3). This means > that the data is stored outside of the cluster and can be accessible after > the cluster has been terminated. One use case that is often asked about is > pointing multiple clusters to one root directory (sharing the data) to have > read resiliency in the case of a cluster failure. > > This JIRA is an umbrella JIRA to contain all the tasks necessary to create a > read-replica HBase cluster that is pointed at the same root directory. > > This requires making the Read-Replica cluster Read-Only (no metadata > operation or data operations). > Separating the hbase:meta table for each cluster (Otherwise HBase gets > confused with multiple clusters trying to update the meta table with their ip > addresses) > Adding refresh functionality for the meta table to ensure new metadata is > picked up on the read replica cluster. > Adding refresh functionality for HFiles for a given table to ensure new data > is picked up on the read replica cluster. > > This can be used with any existing cluster that is backed by an external > filesystem. > > Please note that this feature is still quite manual (with the potential for > automation later). > > More information on this particular feature can be found here: > https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12081: --- Fix Version/s: 1.5.0 1.4.0 > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.5.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175511#comment-16175511 ] Andrew Purtell commented on HBASE-12081: I also added branch-1 and branch-1.4, but I fear that may be optimistic. Will drop branch-1.4 if need be. We can keep this as a blocker for continuing forward with branch-1 (1.5, etc.) > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.5.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[ https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18477: -- Attachment: HBase Read-Replica Clusters Scope doc_v2.pdf Adding the v2 doc as a PDF. [~busbey] sorry for the delay. > Umbrella JIRA for HBase Read Replica clusters > - > > Key: HBASE-18477 > URL: https://issues.apache.org/jira/browse/HBASE-18477 > Project: HBase > Issue Type: New Feature >Reporter: Zach York >Assignee: Zach York > Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase > Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope > doc_v2.docx, HBase Read-Replica Clusters Scope doc_v2.pdf > > > Recently, changes (such as HBASE-17437) have unblocked HBase to run with a > root directory external to the cluster (such as in Amazon S3). This means > that the data is stored outside of the cluster and can be accessible after > the cluster has been terminated. One use case that is often asked about is > pointing multiple clusters to one root directory (sharing the data) to have > read resiliency in the case of a cluster failure. > > This JIRA is an umbrella JIRA to contain all the tasks necessary to create a > read-replica HBase cluster that is pointed at the same root directory. > > This requires making the Read-Replica cluster Read-Only (no metadata > operation or data operations). > Separating the hbase:meta table for each cluster (Otherwise HBase gets > confused with multiple clusters trying to update the meta table with their ip > addresses) > Adding refresh functionality for the meta table to ensure new metadata is > picked up on the read replica cluster. > Adding refresh functionality for HFiles for a given table to ensure new data > is picked up on the read replica cluster. > > This can be used with any existing cluster that is backed by an external > filesystem. > > Please note that this feature is still quite manual (with the potential for > automation later). > > More information on this particular feature can be found here: > https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175515#comment-16175515
]
Hadoop QA commented on HBASE-18847:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-18847 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18847 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888375/HBASE-18847.v1.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8730/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> remove unneeded synchronized block from hfilev2 warning in branch-1.2
> -
>
> Key: HBASE-18847
> URL: https://issues.apache.org/jira/browse/HBASE-18847
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6
>Reporter: Rich Howarth
>Assignee: Rich Howarth
>Priority: Critical
> Fix For: 1.2.7
>
> Attachments: HBASE-18847.v1.patch
>
>
> The below code block starts at line 277 of HFileWriterV2.java. Class-level
> synchronization in a heavily used code path has a demonstrably significant
> negative effect on performance. I tested forcing a major compaction with 18
> compaction threads per node; removing the synchronization resulted in an
> order of magnitude performance increase, with the bottleneck then being at
> the disks (where I want it to be).
> {code:java}
> synchronized (HFileWriterV2.class) {
> if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) {
> LOG.warn("A minimum HFile version of " +
> HFile.MIN_FORMAT_VERSION_WITH_TAGS
> + " is required to support cell attributes/tags. Consider setting "
> + HFile.FORMAT_VERSION_KEY + " accordingly.");
> WARN_CELL_WITH_TAGS = false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175525#comment-16175525
]
Josh Elser commented on HBASE-17732:
Will try to give you a heads-up assuming that HBASE-18807 lands before you get
this one in, [~appy].
Should be a simple conflict to resolve by switching the {{QuotaProtos.Quotas}}
object with the new object {{GlobalQuotaSettings}} (assuming the class name
doesn't change).
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18860) Determine where to move ReadReplicaClustersTableNameUtils
Zach York created HBASE-18860: - Summary: Determine where to move ReadReplicaClustersTableNameUtils Key: HBASE-18860 URL: https://issues.apache.org/jira/browse/HBASE-18860 Project: HBase Issue Type: Sub-task Affects Versions: HBASE-18477 Reporter: Zach York Assignee: Zach York Priority: Minor In HBASE-18773, Stack brought up the (good) point that ReadReplicaClustersTableNameUtils does not really belong in the hbase-common package. This follow-up JIRA is to determine the correct place for Read-Replica code and to move this utils class there. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175536#comment-16175536 ] Zach York commented on HBASE-18773: --- [~stack] I created HBASE-18860 to address where to move this file (and where to put all of the new ReadReplica files). > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Fix For: HBASE-18477 > > Attachments: > 0001-HBASE-18773-Add-utility-method-to-determine-if-a-Tab.patch, > HBASE-18773.HBASE-18477.001.patch, HBASE-18773.HBASE-18477.002.patch, > HBASE-18773.HBASE-18477.003.patch, HBASE-18773.HBASE-18477.004.patch, > HBASE-18773.HBASE-18477.005.patch, HBASE-18773.HBASE-18477.006.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
