[jira] [Created] (HBASE-17314) Limit total buffered size for all replication sources
Phil Yang created HBASE-17314: - Summary: Limit total buffered size for all replication sources Key: HBASE-17314 URL: https://issues.apache.org/jira/browse/HBASE-17314 Project: HBase Issue Type: Bug Components: Replication Reporter: Phil Yang Assignee: Phil Yang If we have many peers or some servers have many recovered queues, we will hold many entries in memory which will increase the pressure of GC, even maybe OOM because we will read entries for 64MB to buffer in default for one source. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747604#comment-15747604 ] Edward Bortnikov commented on HBASE-17081: -- Re/ [~stack]'s question about what's next: 1. HBASE-17294 configuration (Eshcar) - committed, thanks [~stack]. 2. HBASE-16851 documentation (me) - need to complete 3 blog posts: (1) user manual - complete, give or take, (2) performance eval, and (3) programmer's manual. Where should we post all those? Apache blog? 3. HBASE-16417 (Eshcar) automated policy for figuring out whether the BASIC or the EAGER algorithm is to be used. Small refactoring the internal API for future policies. Independent on in-memory compaction per se: 1. HBASE-16421 CellChunkMap implementation (Anastasia) - starting now, need to coordinate with [~anoop.hbase] and [~ram_krish]. 2. JIRA TBD Memstore-First Get (Eshcar) - Big value demonstrated by benchmarks in HBASE-16851, we should try to implement & push before 2.0 closes. Sounds like a plan (smile) ? > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17294) External Configuration for Memory Compaction
[
https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eshcar Hillel updated HBASE-17294:
--
Release Note:
This patch provides a single external knob to control memstore compaction.
Possible memstore compaction policies are:
(1) None - no memory compaction, when size threshold is exceeded data is
flushed to disk
(2) Basic policy applies optimizations which modify the index to a more
compacted representation. This is beneficial in all access patterns. The
smaller the cells are the greater the benefit of this policy. This is the
default policy.
(3) Eager - in addition to compacting the index representation as the basic
policy, eager policy eliminates duplication while the data is still in memory
(much like the on-disk compaction does after the data is flushed to disk). This
policy is most useful for applications with high data churn or small working
sets.
Memory compaction policeman be set at the column family level at table creation
time:
{code}
create ‘’,
{NAME => ‘’,
IN_MEMORY_COMPACTION => ‘’}
{code}
or as a property at the global configuration level by setting the property in
hbase-site.xml, with BASIC being the default value:
{code}
hbase.hregion.compacting.memstore.type
{code}
The values used in this property can change as memstore compaction policies
evolve over time.
> External Configuration for Memory Compaction
> -
>
> Key: HBASE-17294
> URL: https://issues.apache.org/jira/browse/HBASE-17294
> Project: HBase
> Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-17294-V01.patch, HBASE-17294-V02.patch,
> HBASE-17294-V03.patch
>
>
> We would like to have a single external knob to control memstore compaction.
> Possible memstore compaction policies are none, basic, and eager.
> This sub-task allows to set this property at the column family level at table
> creation time:
> {code}
> create ‘’,
>{NAME => ‘’,
> IN_MEMORY_COMPACTION => ‘’}
> {code}
> or to set this at the global configuration level by setting the property in
> hbase-site.xml, with BASIC being the default value:
> {code}
>
> hbase.hregion.compacting.memstore.type
>
>
> {code}
> The values used in this property can change as memstore compaction policies
> evolve over time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-17315) [C++] Implementation of Table Class
Sudeep Sunthankar created HBASE-17315: - Summary: [C++] Implementation of Table Class Key: HBASE-17315 URL: https://issues.apache.org/jira/browse/HBASE-17315 Project: HBase Issue Type: Sub-task Reporter: Sudeep Sunthankar Consists of Table implementation which will be used to call the corresponding client methods i.e Get. MultiGet, Scan etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-17315) [C++] Implementation of Table Class
[ https://issues.apache.org/jira/browse/HBASE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar reassigned HBASE-17315: - Assignee: Sudeep Sunthankar > [C++] Implementation of Table Class > --- > > Key: HBASE-17315 > URL: https://issues.apache.org/jira/browse/HBASE-17315 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > > Consists of Table implementation which will be used to call the corresponding > client methods i.e Get. MultiGet, Scan etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17315) [C++] Implementation of Table Class
[ https://issues.apache.org/jira/browse/HBASE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudeep Sunthankar updated HBASE-17315: -- Attachment: HBASE-17315.HBASE-14850.v1.patch This patch consists of the foll:- # table.h/.cc - Implementation of HTable.java. Will be used to call client operations viz. Get(), Gets(), GetScanner() etc. # protobuf_request_builder.h/.cc - Creates PB request object for the corresponding Scan, Get methods. # Unit-tests hooked up to the client. Thanks. > [C++] Implementation of Table Class > --- > > Key: HBASE-17315 > URL: https://issues.apache.org/jira/browse/HBASE-17315 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17315.HBASE-14850.v1.patch > > > Consists of Table implementation which will be used to call the corresponding > client methods i.e Get. MultiGet, Scan etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17278) Cell Scanner Implementation to be used by ResultScanner
[ https://issues.apache.org/jira/browse/HBASE-17278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747652#comment-15747652 ] Sudeep Sunthankar commented on HBASE-17278: --- Thanks for the feedback [~enis]. I will have a look at the points mentioned above. > Cell Scanner Implementation to be used by ResultScanner > --- > > Key: HBASE-17278 > URL: https://issues.apache.org/jira/browse/HBASE-17278 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Sudeep Sunthankar > Attachments: HBASE-17278.HBASE-14850.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17296: --- Attachment: HBASE-17296-branch-1.patch > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17296: --- Attachment: HBASE-17296-branch-1.patch > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296-branch-1.patch, > HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17316) Addendum to HBASE-17294
Eshcar Hillel created HBASE-17316: - Summary: Addendum to HBASE-17294 Key: HBASE-17316 URL: https://issues.apache.org/jira/browse/HBASE-17316 Project: HBase Issue Type: Sub-task Reporter: Eshcar Hillel Assignee: Eshcar Hillel Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747717#comment-15747717 ] Guanghao Zhang commented on HBASE-17296: Found a bug in ReplicationPeersZKImpl.updatePeerConfig. It didn't update table cfs config and no ut check this... I fix it in this branch-1 patch. But the proper way is open a issue to fix this bug? [~te...@apache.org] > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296-branch-1.patch, > HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17316: -- Attachment: HBASE-17316.patch > Addendum to HBASE-17294 > --- > > Key: HBASE-17316 > URL: https://issues.apache.org/jira/browse/HBASE-17316 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-17316.patch > > > Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17316: -- Status: Patch Available (was: Open) > Addendum to HBASE-17294 > --- > > Key: HBASE-17316 > URL: https://issues.apache.org/jira/browse/HBASE-17316 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-17316.patch > > > Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747734#comment-15747734 ] Eshcar Hillel commented on HBASE-17316: --- Updated the two tests that failed during the commit of HBASE-17294 https://builds.apache.org/job/HBase-Trunk_matrix/2128/ > Addendum to HBASE-17294 > --- > > Key: HBASE-17316 > URL: https://issues.apache.org/jira/browse/HBASE-17316 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-17316.patch > > > Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17294) External Configuration for Memory Compaction
[
https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747741#comment-15747741
]
Eshcar Hillel commented on HBASE-17294:
---
Opened HBASE-17316 with a new patch which updates the two tests that failed
during the Jenkins build.
> External Configuration for Memory Compaction
> -
>
> Key: HBASE-17294
> URL: https://issues.apache.org/jira/browse/HBASE-17294
> Project: HBase
> Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-17294-V01.patch, HBASE-17294-V02.patch,
> HBASE-17294-V03.patch
>
>
> We would like to have a single external knob to control memstore compaction.
> Possible memstore compaction policies are none, basic, and eager.
> This sub-task allows to set this property at the column family level at table
> creation time:
> {code}
> create ‘’,
>{NAME => ‘’,
> IN_MEMORY_COMPACTION => ‘’}
> {code}
> or to set this at the global configuration level by setting the property in
> hbase-site.xml, with BASIC being the default value:
> {code}
>
> hbase.hregion.compacting.memstore.type
>
>
> {code}
> The values used in this property can change as memstore compaction policies
> evolve over time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747744#comment-15747744 ] Anastasia Braginsky commented on HBASE-17081: - [~stack], thank you for your comments, here are my answers: bq. Are we going to have an if/else per flush policy We would prefer to leave the if-else statement whether to flush everything or not. Although currently we always flushing everything (using composite snapshot). Under HBASE-16417, Eshcar is still running more experiments to find the best policy. So maybe it will be needed there. If not, it can be removed as part of HBASE-16417. bq. Can you presize the Array? Yes, I can presize the Array. However it would be a pity, to create now a new patch just for that. Please let me add this in using my next JIRA. bq. As said before, this is expensive... TimeRangeTracker is requested by MemStoreSnapshot and is later used by the DefaultMobStoreFlusher. Maybe it can be anyhow avoided, but I didn't look so deep. As TimeRangeTracker was initially part of the snapshot, I left it there. It can be removed by another JIRA if indeed unneeded... [~anoop.hbase], it looks like everybody are waiting for your blessing! :) > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[
https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747766#comment-15747766
]
Hadoop QA commented on HBASE-17081:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 15s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 90m 44s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 44s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843165/HBASE-15787_8.patch |
| JIRA Issue | HBASE-17081 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux ab464132823e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / a73b0b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/4909/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/4909/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4909/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4909/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Flush the entire CompactingMemStore content to disk
> ---
>
>
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747790#comment-15747790 ] Anastasia Braginsky commented on HBASE-17081: - I didn't submitted HBASE-15787_8.patch patch here... ?! Is it any error? [~ram_krish] did you intentionally added it here? > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17291) Remove ImmutableSegment#getKeyValueScanner
[ https://issues.apache.org/jira/browse/HBASE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747818#comment-15747818 ] Anastasia Braginsky commented on HBASE-17291: - [~ram_krish], please add your patch to the review board. Can you also please elaborate about "all the test cases in HBASE-17081 passes"? Currently in all tests in HBASE-17081 are OK, is it that they do not pass with this patch? > Remove ImmutableSegment#getKeyValueScanner > -- > > Key: HBASE-17291 > URL: https://issues.apache.org/jira/browse/HBASE-17291 > Project: HBase > Issue Type: Improvement > Components: Scanners >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17291.patch, HBASE-17291_1.patch > > > This is based on a discussion over [~anastas]'s patch. The MemstoreSnapshot > uses a KeyValueScanner which actually seems redundant considering we already > have a SegmentScanner. The idea is that the snapshot scanner should be a > simple iterator type of scanner but it lacks the capability to do the > reference counting on that segment that is now used in snapshot. With > snapshot having mulitple segments in the latest impl it is better we hold on > to the segment by doing ref counting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747838#comment-15747838 ] Ted Yu commented on HBASE-17296: Yes, you can open a new issue to fix the bug. > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296-branch-1.patch, > HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15787:
---
Status: Open (was: Patch Available)
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch,
> HBASE-15787_4.patch, HBASE-15787_5.patch, HBASE-15787_6.patch,
> HBASE-15787_7.patch
>
>
> With offheap MSLAB in place we may have to change the flush related
> heuristics to work with offheap size configured rather than the java heap
> size.
> Since we now have clear seperation of the memstore data size and memstore
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking
> updates and force flushes.
> -> If the onheap global.memstore.size is breached (due to heap overhead) even
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the
> global.memstore.size. So now we apply this 95% on the
> global.offheap.memstore.size and also on global.memstore.size (as it was done
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
> ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_lower_mark*/
> ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_higher_mark*/
> ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap
> memstore settings*/
> ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that
> at no point it will try to increase or decrease the
> global.offheap.memstore.size. If there is a heap pressure then it will try to
> increase the memstore heap limit.
> In case of offheap memstore there is always a chance that the heap pressure
> does not increase. In that case we could ideally decrease the heap limit for
> memstore. The current logic of heapmemory tuner is such that things will
> naturally settle down. But on discussion what we thought is let us include
> the flush count that happens due to offheap pressure but give that a lesser
> weightage and thus ensure that the initial decrease on memstore heap limit
> does not happen. Currently that fraction is set as 0.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15787:
---
Attachment: HBASE-15787_8.patch
Attaching the right patch.
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch,
> HBASE-15787_4.patch, HBASE-15787_5.patch, HBASE-15787_6.patch,
> HBASE-15787_7.patch, HBASE-15787_8.patch
>
>
> With offheap MSLAB in place we may have to change the flush related
> heuristics to work with offheap size configured rather than the java heap
> size.
> Since we now have clear seperation of the memstore data size and memstore
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking
> updates and force flushes.
> -> If the onheap global.memstore.size is breached (due to heap overhead) even
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the
> global.memstore.size. So now we apply this 95% on the
> global.offheap.memstore.size and also on global.memstore.size (as it was done
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
> ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_lower_mark*/
> ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_higher_mark*/
> ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap
> memstore settings*/
> ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that
> at no point it will try to increase or decrease the
> global.offheap.memstore.size. If there is a heap pressure then it will try to
> increase the memstore heap limit.
> In case of offheap memstore there is always a chance that the heap pressure
> does not increase. In that case we could ideally decrease the heap limit for
> memstore. The current logic of heapmemory tuner is such that things will
> naturally settle down. But on discussion what we thought is let us include
> the flush count that happens due to offheap pressure but give that a lesser
> weightage and thus ensure that the initial decrease on memstore heap limit
> does not happen. Currently that fraction is set as 0.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15787:
---
Status: Patch Available (was: Open)
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch,
> HBASE-15787_4.patch, HBASE-15787_5.patch, HBASE-15787_6.patch,
> HBASE-15787_7.patch, HBASE-15787_8.patch
>
>
> With offheap MSLAB in place we may have to change the flush related
> heuristics to work with offheap size configured rather than the java heap
> size.
> Since we now have clear seperation of the memstore data size and memstore
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking
> updates and force flushes.
> -> If the onheap global.memstore.size is breached (due to heap overhead) even
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the
> global.memstore.size. So now we apply this 95% on the
> global.offheap.memstore.size and also on global.memstore.size (as it was done
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
> ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_lower_mark*/
> ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap
> memstore settings
> An offheap memstore can even breach the
> onheap_higher_mark*/
> ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap
> memstore settings*/
> ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that
> at no point it will try to increase or decrease the
> global.offheap.memstore.size. If there is a heap pressure then it will try to
> increase the memstore heap limit.
> In case of offheap memstore there is always a chance that the heap pressure
> does not increase. In that case we could ideally decrease the heap limit for
> memstore. The current logic of heapmemory tuner is such that things will
> naturally settle down. But on discussion what we thought is let us include
> the flush count that happens due to offheap pressure but give that a lesser
> weightage and thus ensure that the initial decrease on memstore heap limit
> does not happen. Currently that fraction is set as 0.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747854#comment-15747854 ] ramkrishna.s.vasudevan commented on HBASE-17081: Oops sorry. I was having both JIRAs opened and had a server glitch and so added in a wrong one. > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17291) Remove ImmutableSegment#getKeyValueScanner
[ https://issues.apache.org/jira/browse/HBASE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747858#comment-15747858 ] ramkrishna.s.vasudevan commented on HBASE-17291: I meant that once HBASE-17081 is committed I will run this patch against that change. And then ensure all tests pass. > Remove ImmutableSegment#getKeyValueScanner > -- > > Key: HBASE-17291 > URL: https://issues.apache.org/jira/browse/HBASE-17291 > Project: HBase > Issue Type: Improvement > Components: Scanners >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17291.patch, HBASE-17291_1.patch > > > This is based on a discussion over [~anastas]'s patch. The MemstoreSnapshot > uses a KeyValueScanner which actually seems redundant considering we already > have a SegmentScanner. The idea is that the snapshot scanner should be a > simple iterator type of scanner but it lacks the capability to do the > reference counting on that segment that is now used in snapshot. With > snapshot having mulitple segments in the latest impl it is better we hold on > to the segment by doing ref counting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17296: --- Attachment: HBASE-17296-branch-1.patch > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296-branch-1.patch, > HBASE-17296-branch-1.patch, HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747875#comment-15747875 ] Guanghao Zhang commented on HBASE-17296: Attach a new patch for branch-1 which didn't include the fix. > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296-branch-1.patch, > HBASE-17296-branch-1.patch, HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17291) Remove ImmutableSegment#getKeyValueScanner
[ https://issues.apache.org/jira/browse/HBASE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747888#comment-15747888 ] Anastasia Braginsky commented on HBASE-17291: - Sure! Also then you will rebase and put on the review board? > Remove ImmutableSegment#getKeyValueScanner > -- > > Key: HBASE-17291 > URL: https://issues.apache.org/jira/browse/HBASE-17291 > Project: HBase > Issue Type: Improvement > Components: Scanners >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-17291.patch, HBASE-17291_1.patch > > > This is based on a discussion over [~anastas]'s patch. The MemstoreSnapshot > uses a KeyValueScanner which actually seems redundant considering we already > have a SegmentScanner. The idea is that the snapshot scanner should be a > simple iterator type of scanner but it lacks the capability to do the > reference counting on that segment that is now used in snapshot. With > snapshot having mulitple segments in the latest impl it is better we hold on > to the segment by doing ref counting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17317) [branch-1] The updatePeerConfig method in ReplicationPeersZKImpl didn't update the table-cfs map
Guanghao Zhang created HBASE-17317:
--
Summary: [branch-1] The updatePeerConfig method in
ReplicationPeersZKImpl didn't update the table-cfs map
Key: HBASE-17317
URL: https://issues.apache.org/jira/browse/HBASE-17317
Project: HBase
Issue Type: Task
Affects Versions: 1.4.0
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang
The updatePeerConfig method in ReplicationPeersZKImpl.java
{code}
@Override
public void updatePeerConfig(String id, ReplicationPeerConfig newConfig)
throws ReplicationException {
ReplicationPeer peer = getPeer(id);
if (peer == null){
throw new ReplicationException("Could not find peer Id " + id);
}
ReplicationPeerConfig existingConfig = peer.getPeerConfig();
if (newConfig.getClusterKey() != null &&
!newConfig.getClusterKey().isEmpty() &&
!newConfig.getClusterKey().equals(existingConfig.getClusterKey())){
throw new ReplicationException("Changing the cluster key on an existing
peer is not allowed."
+ " Existing key '" + existingConfig.getClusterKey() + "' does not
match new key '"
+ newConfig.getClusterKey() +
"'");
}
String existingEndpointImpl = existingConfig.getReplicationEndpointImpl();
if (newConfig.getReplicationEndpointImpl() != null &&
!newConfig.getReplicationEndpointImpl().isEmpty() &&
!newConfig.getReplicationEndpointImpl().equals(existingEndpointImpl)){
throw new ReplicationException("Changing the replication endpoint
implementation class " +
"on an existing peer is not allowed. Existing class '"
+ existingConfig.getReplicationEndpointImpl()
+ "' does not match new class '" +
newConfig.getReplicationEndpointImpl() + "'");
}
//Update existingConfig's peer config and peer data with the new values,
but don't touch config
// or data that weren't explicitly changed
existingConfig.getConfiguration().putAll(newConfig.getConfiguration());
existingConfig.getPeerData().putAll(newConfig.getPeerData());
// Bug. We should update table-cfs map, too.
try {
ZKUtil.setData(this.zookeeper, getPeerNode(id),
ReplicationSerDeHelper.toByteArray(existingConfig));
}
catch(KeeperException ke){
throw new ReplicationException("There was a problem trying to save
changes to the " +
"replication peer " + id, ke);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16398) optimize HRegion computeHDFSBlocksDistribution
[ https://issues.apache.org/jira/browse/HBASE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747917#comment-15747917 ] Ted Yu commented on HBASE-16398: lgtm > optimize HRegion computeHDFSBlocksDistribution > -- > > Key: HBASE-16398 > URL: https://issues.apache.org/jira/browse/HBASE-16398 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.0.0 >Reporter: binlijin >Assignee: binlijin > Fix For: 2.0.0 > > Attachments: HBASE-16398.patch, HBASE-16398_v2.patch, > HBASE-16398_v3.patch, HBASE-16398_v4.patch, HBASE-16398_v5.patch, > LocatedBlockStatusComparison.java > > > First i assume there is no reference and link in a region family's directory. > Without the patch to computeHDFSBlocksDistribution for a region family, there > is 1+2*N rpc call, N is hfile numbers, The first rpc call is to > DistributedFileSystem#listStatus to get hfiles, for every hfile there is two > rpc call DistributedFileSystem#getFileStatus(path) and then > DistributedFileSystem#getFileBlockLocations(status, start, length). > With the patch to computeHDFSBlocksDistribution for a region family, there is > 2 rpc call, they are DistributedFileSystem#getFileStatus(path) and > DistributedFileSystem#listLocatedStatus(final Path p, final PathFilter > filter). > So if there is at least one hfile, with the patch, the rpc call will less. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747938#comment-15747938 ] Guanghao Zhang commented on HBASE-17296: Open a new issue HBASE-17317 for the bug. > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296-branch-1.patch, > HBASE-17296-branch-1.patch, HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11392) add/remove peer requests should be routed through master
[
https://issues.apache.org/jira/browse/HBASE-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747939#comment-15747939
]
Hadoop QA commented on HBASE-11392:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 50s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 45s
{color} | {color:red} hbase-client in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 15 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 27s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 1m
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 55s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 159m 13s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843174/HBASE-11392-v2.patch |
| JIRA Issue | HBASE-11392 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile cc hbaseprotoc |
| uname | Linux a17a7144a6e4 3.13.0-103-generic #
[jira] [Commented] (HBASE-11392) add/remove peer requests should be routed through master
[ https://issues.apache.org/jira/browse/HBASE-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747966#comment-15747966 ] Guanghao Zhang commented on HBASE-11392: The hbase-client findbugs warning "org.apache.hadoop.hbase.client.BufferedMutatorParams defines clone() but doesn't implement Cloneable" is not related to this patch. > add/remove peer requests should be routed through master > > > Key: HBASE-11392 > URL: https://issues.apache.org/jira/browse/HBASE-11392 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Guanghao Zhang >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-11392-v1.patch, HBASE-11392-v2.patch > > > ReplicationAdmin directly operates over the zookeeper data for replication > setup. We should move these operations to be routed through master for two > reasons: > - Replication implementation details are exposed to client. We should move > most of the replication related classes to hbase-server package. > - Routing the requests through master is the standard practice for all other > operations. It allows for decoupling implementation details from the client > and code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9774) Provide a way for coprocessors to register and report custom metrics
[
https://issues.apache.org/jira/browse/HBASE-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748020#comment-15748020
]
Hadoop QA commented on HBASE-9774:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m 55s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 13 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 42s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
59s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: .
hbase-assembly {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 58s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 5s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 9s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
24m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: .
hbase-assembly {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 9s
{color} | {color:green} hbase-metrics-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 56s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 13s
{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s
{color} | {color:green} hbase-metrics in the pat
[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294
[
https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748029#comment-15748029
]
Hadoop QA commented on HBASE-17316:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 27s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 38s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843190/HBASE-17316.patch |
| JIRA Issue | HBASE-17316 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5254148747f5 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a73b0b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4912/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4912/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Addendum to HBASE-17294
> ---
>
> Key: HBASE-17316
> URL: https://issues.apache.org/jira/browse/HBASE-17316
> Project: HBase
> Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Attachments: HBASE-17316.patch
>
>
> Updating 2 tests that failed during the commit of HBASE-17294
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748035#comment-15748035 ] Eshcar Hillel commented on HBASE-17316: --- QA passed. [~stack] please consider committing this addendum as well. > Addendum to HBASE-17294 > --- > > Key: HBASE-17316 > URL: https://issues.apache.org/jira/browse/HBASE-17316 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-17316.patch > > > Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16421) Introducing the CellChunkMap as a new additional index variant in the MemStore
[ https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748066#comment-15748066 ] Anastasia Braginsky commented on HBASE-16421: - Returning to this issue after a while. [~anoop.hbase], [~ram_krish], do you (by any chance) have some results about running scans on top of CellChunkMaps? (related to the possible GC problem with scan's performance as discussed earlier) > Introducing the CellChunkMap as a new additional index variant in the MemStore > -- > > Key: HBASE-16421 > URL: https://issues.apache.org/jira/browse/HBASE-16421 > Project: HBase > Issue Type: Umbrella >Reporter: Anastasia Braginsky > Attachments: CellChunkMapRevived.pdf, > IntroductiontoNewFlatandCompactMemStore.pdf > > > Follow up for HBASE-14921. This is going to be the umbrella JIRA to include > all the parts of integration of the CellChunkMap to the MemStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication
[
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748115#comment-15748115
]
Hadoop QA commented on HBASE-17296:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 38s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 1s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 1s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 13s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
55s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m
48s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
4s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 51s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 53s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 85m 51s
{color} | {color:green} hbase-server in the patch
[jira] [Commented] (HBASE-16421) Introducing the CellChunkMap as a new additional index variant in the MemStore
[ https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748154#comment-15748154 ] ramkrishna.s.vasudevan commented on HBASE-16421: I have not done that. But let me get back to you before the end of this year with my findings. I will complete that AR ASAP. > Introducing the CellChunkMap as a new additional index variant in the MemStore > -- > > Key: HBASE-16421 > URL: https://issues.apache.org/jira/browse/HBASE-16421 > Project: HBase > Issue Type: Umbrella >Reporter: Anastasia Braginsky > Attachments: CellChunkMapRevived.pdf, > IntroductiontoNewFlatandCompactMemStore.pdf > > > Follow up for HBASE-14921. This is going to be the umbrella JIRA to include > all the parts of integration of the CellChunkMap to the MemStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15787) Change the flush related heuristics to work with offheap size configured
[
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748174#comment-15748174
]
Hadoop QA commented on HBASE-15787:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m 52s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 126m 10s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843197/HBASE-15787_8.patch |
| JIRA Issue | HBASE-15787 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 7267434fa456 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / a73b0b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4914/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4914/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Change the flush related heuristics to work with offheap size configured
>
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0
[jira] [Commented] (HBASE-17313) Add BufferedMutatorParams#clone method
[ https://issues.apache.org/jira/browse/HBASE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748219#comment-15748219 ] Hudson commented on HBASE-17313: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2130 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2130/]) HBASE-17313 Add BufferedMutatorParams#clone method (Joep Rottinghuis) (stack: rev a73b0b3e6d42c3f7fd0c3a0d9f80c2763a6cf069) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorParams.java * (add) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutatorParams.java > Add BufferedMutatorParams#clone method > -- > > Key: HBASE-17313 > URL: https://issues.apache.org/jira/browse/HBASE-17313 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Fix For: 2.0.0 > > Attachments: HBASE-17313.master.001.patch, > HBASE-17313.master.002.patch > > > In HBASE-17277 we allow an alternate BufferedMutator implementation. If we > want to use that and not make BufferedMutatorImpl public, we need to be able > to ask the connection to create us one. In that case, we need to clone the > params (and then strip one argument off). > This jira is to add a clone method for convenience. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17277) Allow alternate BufferedMutator implementation
[ https://issues.apache.org/jira/browse/HBASE-17277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748221#comment-15748221 ] Hudson commented on HBASE-17277: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2130 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2130/]) HBASE-17277 Allow alternate BufferedMutator implemenation Specify the (stack: rev 68ce3f1e3b22e8a8c6b17072738be6a08489dfca) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutator.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorParams.java * (add) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutator.java > Allow alternate BufferedMutator implementation > -- > > Key: HBASE-17277 > URL: https://issues.apache.org/jira/browse/HBASE-17277 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-17277.master.001.patch, > HBASE-17277.master.002.patch > > > Allow being able to supply alternate BufferedMutator implementation. For > example, the parent issue would like to spool writes to the filesystem if > hbase is down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication
[
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748224#comment-15748224
]
Hadoop QA commented on HBASE-17296:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m 59s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 14s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
55s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
5s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
3s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 52s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 56s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 49s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 116m 24s
{color} | {color:red} hbase-server in the patch failed
[jira] [Created] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing
Guangxu Cheng created HBASE-17318: - Summary: Increment does not add new column if the increment amount is zero at first time writing Key: HBASE-17318 URL: https://issues.apache.org/jira/browse/HBASE-17318 Project: HBase Issue Type: Improvement Affects Versions: 1.2.4, 0.98.23, 2.0.0 Reporter: Guangxu Cheng When the data written for the first time is 0, no new columns are added. Iterate the input columns and update existing values if they were found, otherwise add new column initialized to the increment amount. Does not add new column if the increment amount is zero at first time writting. It is necessary to add a new column at the first write to 0. If not, the result of using the phoenix is NULL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing
[ https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guangxu Cheng updated HBASE-17318: -- Attachment: HBASE-17318-branch-1.2-v1.patch > Increment does not add new column if the increment amount is zero at first > time writing > --- > > Key: HBASE-17318 > URL: https://issues.apache.org/jira/browse/HBASE-17318 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 0.98.23, 1.2.4 >Reporter: Guangxu Cheng > Attachments: HBASE-17318-branch-1.2-v1.patch > > > When the data written for the first time is 0, no new columns are added. > Iterate the input columns and update existing values if they were found, > otherwise add new column initialized to the increment amount. > Does not add new column if the increment amount is zero at first time > writting. > It is necessary to add a new column at the first write to 0. > If not, the result of using the phoenix is NULL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication
[
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748320#comment-15748320
]
Hadoop QA commented on HBASE-17296:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 1s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 1s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 31s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
2s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
51s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 59s
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 12s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 59s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 113m 42s
{color} | {color:red} hbase-server in the patch faile
[jira] [Updated] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing
[ https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guangxu Cheng updated HBASE-17318: -- Status: Patch Available (was: Open) > Increment does not add new column if the increment amount is zero at first > time writing > --- > > Key: HBASE-17318 > URL: https://issues.apache.org/jira/browse/HBASE-17318 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.4, 0.98.23, 2.0.0 >Reporter: Guangxu Cheng > Attachments: HBASE-17318-branch-1.2-v1.patch > > > When the data written for the first time is 0, no new columns are added. > Iterate the input columns and update existing values if they were found, > otherwise add new column initialized to the increment amount. > Does not add new column if the increment amount is zero at first time > writting. > It is necessary to add a new column at the first write to 0. > If not, the result of using the phoenix is NULL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing
[ https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748594#comment-15748594 ] Ted Yu commented on HBASE-17318: Can you add a unit test to prevent regression ? Please attach patch for master branch. > Increment does not add new column if the increment amount is zero at first > time writing > --- > > Key: HBASE-17318 > URL: https://issues.apache.org/jira/browse/HBASE-17318 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 0.98.23, 1.2.4 >Reporter: Guangxu Cheng > Attachments: HBASE-17318-branch-1.2-v1.patch > > > When the data written for the first time is 0, no new columns are added. > Iterate the input columns and update existing values if they were found, > otherwise add new column initialized to the increment amount. > Does not add new column if the increment amount is zero at first time > writting. > It is necessary to add a new column at the first write to 0. > If not, the result of using the phoenix is NULL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees
[
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748619#comment-15748619
]
Ted Yu commented on HBASE-17309:
{code}
148 table.close();
149 connection.close();
{code}
Please enclose connection.close() in finally block.
> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17309-master-001.patch
>
>
> Connections are not closed in some of TestAcidGuarantees's classes, this
> makes integration test fail in some cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17313) Add BufferedMutatorParams#clone method
[ https://issues.apache.org/jira/browse/HBASE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748738#comment-15748738 ] stack commented on HBASE-17313: --- Thanks [~jrottinghuis]. I pushed the amendment HBASE-17313.master.002.patch > Add BufferedMutatorParams#clone method > -- > > Key: HBASE-17313 > URL: https://issues.apache.org/jira/browse/HBASE-17313 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Fix For: 2.0.0 > > Attachments: HBASE-17313.master.001.patch, > HBASE-17313.master.002.patch > > > In HBASE-17277 we allow an alternate BufferedMutator implementation. If we > want to use that and not make BufferedMutatorImpl public, we need to be able > to ask the connection to create us one. In that case, we need to clone the > params (and then strip one argument off). > This jira is to add a clone method for convenience. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294, 'External Configuration for Memory Compaction'
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17316: -- Summary: Addendum to HBASE-17294, 'External Configuration for Memory Compaction' (was: Addendum to HBASE-17294) > Addendum to HBASE-17294, 'External Configuration for Memory Compaction' > --- > > Key: HBASE-17316 > URL: https://issues.apache.org/jira/browse/HBASE-17316 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-17316.patch > > > Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294, 'External Configuration for Memory Compaction'
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17316: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks @eshcar for keeping an eye on builds fixing my breaking commit. > Addendum to HBASE-17294, 'External Configuration for Memory Compaction' > --- > > Key: HBASE-17316 > URL: https://issues.apache.org/jira/browse/HBASE-17316 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 2.0.0 > > Attachments: HBASE-17316.patch > > > Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17018) Spooling BufferedMutator
[ https://issues.apache.org/jira/browse/HBASE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748827#comment-15748827 ] Joep Rottinghuis commented on HBASE-17018: -- If I can make it so that the SpoolingBufferedMutator doesn't even connect to an actual BufferedMutatorImpl, this setup could help in some tests where you want to be able to send mutations but just have them appear in a file. > Spooling BufferedMutator > > > Key: HBASE-17018 > URL: https://issues.apache.org/jira/browse/HBASE-17018 > Project: HBase > Issue Type: New Feature >Reporter: Joep Rottinghuis > Attachments: HBASE-17018.master.001.patch, > HBASE-17018SpoolingBufferedMutatorDesign-v1.pdf, YARN-4061 HBase requirements > for fault tolerant writer.pdf > > > For Yarn Timeline Service v2 we use HBase as a backing store. > A big concern we would like to address is what to do if HBase is > (temporarily) down, for example in case of an HBase upgrade. > Most of the high volume writes will be mostly on a best-effort basis, but > occasionally we do a flush. Mainly during application lifecycle events, > clients will call a flush on the timeline service API. In order to handle the > volume of writes we use a BufferedMutator. When flush gets called on our API, > we in turn call flush on the BufferedMutator. > We would like our interface to HBase be able to spool the mutations to a > filesystems in case of HBase errors. If we use the Hadoop filesystem > interface, this can then be HDFS, gcs, s3, or any other distributed storage. > The mutations can then later be re-played, for example through a MapReduce > job. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing
[
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748877#comment-15748877
]
Hadoop QA commented on HBASE-17318:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 44s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 44s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 108m 7s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.wal.TestDurability |
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:d6626eb |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843220/HBASE-17318-branch-1.2-v1.patch
|
| JIRA Issue | HBASE-17318 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f0bef923c65d 3.13.0-93
[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing
[ https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749019#comment-15749019 ] Ted Yu commented on HBASE-17318: Failure of TestDurability might be related to the patch. > Increment does not add new column if the increment amount is zero at first > time writing > --- > > Key: HBASE-17318 > URL: https://issues.apache.org/jira/browse/HBASE-17318 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 0.98.23, 1.2.4 >Reporter: Guangxu Cheng > Attachments: HBASE-17318-branch-1.2-v1.patch > > > When the data written for the first time is 0, no new columns are added. > Iterate the input columns and update existing values if they were found, > otherwise add new column initialized to the increment amount. > Does not add new column if the increment amount is zero at first time > writting. > It is necessary to add a new column at the first write to 0. > If not, the result of using the phoenix is NULL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749072#comment-15749072 ] stack commented on HBASE-17081: --- Apache blog and refguide would be great places for blog and operator/dev doc. > Flush the entire CompactingMemStore content to disk > --- > > Key: HBASE-17081 > URL: https://issues.apache.org/jira/browse/HBASE-17081 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, > HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, > HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, > HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch > > > Part of CompactingMemStore's memory is held by an active segment, and another > part is divided between immutable segments in the compacting pipeline. Upon > flush-to-disk request we want to flush all of it to disk, in contrast to > flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16579) Concurrent backup / restore support
[ https://issues.apache.org/jira/browse/HBASE-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-16579. Resolution: Later With dramatic refactoring, this may require significant amount of work. > Concurrent backup / restore support > --- > > Key: HBASE-16579 > URL: https://issues.apache.org/jira/browse/HBASE-16579 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > Labels: backup > > Currently we take exclusive lock on hbase:backup table in backup / restore > procedures. > There is only one backup or restore operation which can run at given time. > We should consider using shared lock on hbase:backup table in the procedures > so that concurrent backup / restore operations can proceed. > The concurrency is useful when table sets don't overlap. > We may devise other mechanism(s) when there is overlap among the table sets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16542) Skip full backup in selected backup tests
[ https://issues.apache.org/jira/browse/HBASE-16542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749099#comment-15749099 ] Ted Yu commented on HBASE-16542: Take TestIncrementalBackupDeleteTable as an example, skipping the restore of full backup reduced runtime by 1 minute. > Skip full backup in selected backup tests > - > > Key: HBASE-16542 > URL: https://issues.apache.org/jira/browse/HBASE-16542 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Priority: Minor > Labels: backup > > Since automatic mode is always used (HBASE-16037), some tests take longer > time to run: > 1. restore full backup > 2. restore incremental backup > Action 2 would execute action 1 again. > We can selectively skip full backup in backup / restore tests where > incremental backup is involved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17149) Procedure v2 - Fix nonce submission
[ https://issues.apache.org/jira/browse/HBASE-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17149: -- Attachment: nonce.patch Matteo's patch > Procedure v2 - Fix nonce submission > --- > > Key: HBASE-17149 > URL: https://issues.apache.org/jira/browse/HBASE-17149 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 1.2.4 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Attachments: nonce.patch > > > instead of having all the logic in submitProcedure(), split in > registerNonce() + submitProcedure(). > In this case we can avoid calling the coprocessor twice and having a clean > submit logic knowing that there will only be one submission. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17149) Procedure v2 - Fix nonce submission
[ https://issues.apache.org/jira/browse/HBASE-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17149: -- Status: Patch Available (was: Open) > Procedure v2 - Fix nonce submission > --- > > Key: HBASE-17149 > URL: https://issues.apache.org/jira/browse/HBASE-17149 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 1.2.4, 1.1.7, 2.0.0, 1.3.0, 1.4.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Attachments: nonce.patch > > > instead of having all the logic in submitProcedure(), split in > registerNonce() + submitProcedure(). > In this case we can avoid calling the coprocessor twice and having a clean > submit logic knowing that there will only be one submission. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master
[ https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17000: --- Attachment: HBASE-17000.005.HBASE-16961.patch .005 encompasses all suggestions from reviewboard. > [RegionServer] Compute region filesystem space use and report to Master > --- > > Key: HBASE-17000 > URL: https://issues.apache.org/jira/browse/HBASE-17000 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-17000.001.patch, > HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch, > HBASE-17000.005.HBASE-16961.patch > > > Each RegionServer needs to track how much space a Region takes up and roll > this up to the table level. > Aggregation of this information in the Master will be covered by HBASE-16997. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749199#comment-15749199
]
Vladimir Rodionov commented on HBASE-17292:
---
Ted, couple notes:
# Hooks in RegionObserver usually comes in pairs: pre- and post-. You introduce
only one.
# Who else is going to use this hook in your opinion?
{code}
/**
* Called before moving bulk loaded hfile to region directory.
*
* @param ctx
* @param family column family
* @param pairs List of pairs of { HFile location in staging dir, HFile path
in region dir }
* Each pair are for the same hfile.
* @throws IOException
*/
void preCommitStoreFile(final ObserverContext
ctx,
final byte[] family, final List> pairs) throws
IOException;
{code}
You have String and Path for file locations. Make them both Path.
> Add observer notification before bulk loaded hfile is moved to region
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but
> before postBulkLoadHFile() hook is called, there is no way for pluggable
> components (replication, backup / restore) to know which hfile(s) have been
> moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be
> committed hfile before bulk loaded hfile is moved to region directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749233#comment-15749233 ] Vladimir Rodionov commented on HBASE-17292: --- Another note: For your specific problem you extend public API (RegionObserver). Make sure you have other use cases for new API and you can convince community in necessity of this decision. > Add observer notification before bulk loaded hfile is moved to region > directory > --- > > Key: HBASE-17292 > URL: https://issues.apache.org/jira/browse/HBASE-17292 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17292.v1.txt > > > Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded > hfiles. > However, if bulk load fails after hfile is moved to region directory but > before postBulkLoadHFile() hook is called, there is no way for pluggable > components (replication, backup / restore) to know which hfile(s) have been > moved to region directory. > This issue adds a preCommitStoreFile() hook which notifies path of to be > committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749244#comment-15749244
]
Ted Yu commented on HBASE-17292:
bq. Hooks in RegionObserver usually comes in pairs: pre- and post-
Just want to confirm: you're suggesting adding postCommitStoreFile() in
HStore#bulkLoadHFile() after fs.commitStoreFile() call:
{code}
public Path bulkLoadHFile(String srcPathStr, Path dstPath) throws IOException
{
Path srcPath = new Path(srcPathStr);
fs.commitStoreFile(srcPath, dstPath);
{code}
bq. Who else is going to use this hook
Once this hook is added, bulk load support for replication would be able to use
it as well.
> Add observer notification before bulk loaded hfile is moved to region
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but
> before postBulkLoadHFile() hook is called, there is no way for pluggable
> components (replication, backup / restore) to know which hfile(s) have been
> moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be
> committed hfile before bulk loaded hfile is moved to region directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17292: --- Description: Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded hfiles. However, if bulk load fails after hfile is moved to region directory but before postBulkLoadHFile() hook is called, there is no way for pluggable components (replication - see HBASE-17290, backup / restore) to know which hfile(s) have been moved to region directory. This issue adds a preCommitStoreFile() hook which notifies path of to be committed hfile before bulk loaded hfile is moved to region directory. was: Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded hfiles. However, if bulk load fails after hfile is moved to region directory but before postBulkLoadHFile() hook is called, there is no way for pluggable components (replication, backup / restore) to know which hfile(s) have been moved to region directory. This issue adds a preCommitStoreFile() hook which notifies path of to be committed hfile before bulk loaded hfile is moved to region directory. > Add observer notification before bulk loaded hfile is moved to region > directory > --- > > Key: HBASE-17292 > URL: https://issues.apache.org/jira/browse/HBASE-17292 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17292.v1.txt > > > Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded > hfiles. > However, if bulk load fails after hfile is moved to region directory but > before postBulkLoadHFile() hook is called, there is no way for pluggable > components (replication - see HBASE-17290, backup / restore) to know which > hfile(s) have been moved to region directory. > This issue adds a preCommitStoreFile() hook which notifies path of to be > committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749297#comment-15749297 ] Vladimir Rodionov commented on HBASE-17292: --- Yes, add postCommit hook and fix . lgtm after that. > Add observer notification before bulk loaded hfile is moved to region > directory > --- > > Key: HBASE-17292 > URL: https://issues.apache.org/jira/browse/HBASE-17292 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17292.v1.txt > > > Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded > hfiles. > However, if bulk load fails after hfile is moved to region directory but > before postBulkLoadHFile() hook is called, there is no way for pluggable > components (replication - see HBASE-17290, backup / restore) to know which > hfile(s) have been moved to region directory. > This issue adds a preCommitStoreFile() hook which notifies path of to be > committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17294) External Configuration for Memory Compaction
[
https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749367#comment-15749367
]
Hudson commented on HBASE-17294:
SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2132 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/2132/])
HBASE-17316: Addendum of HBASE-17294 (stack: rev
691f266fc2f2b3d7bd9b3108f795f12b35e334c3)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
> External Configuration for Memory Compaction
> -
>
> Key: HBASE-17294
> URL: https://issues.apache.org/jira/browse/HBASE-17294
> Project: HBase
> Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-17294-V01.patch, HBASE-17294-V02.patch,
> HBASE-17294-V03.patch
>
>
> We would like to have a single external knob to control memstore compaction.
> Possible memstore compaction policies are none, basic, and eager.
> This sub-task allows to set this property at the column family level at table
> creation time:
> {code}
> create ‘’,
>{NAME => ‘’,
> IN_MEMORY_COMPACTION => ‘’}
> {code}
> or to set this at the global configuration level by setting the property in
> hbase-site.xml, with BASIC being the default value:
> {code}
>
> hbase.hregion.compacting.memstore.type
>
>
> {code}
> The values used in this property can change as memstore compaction policies
> evolve over time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17313) Add BufferedMutatorParams#clone method
[ https://issues.apache.org/jira/browse/HBASE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749373#comment-15749373 ] Hudson commented on HBASE-17313: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2132 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2132/]) HBASE-17313 Add BufferedMutatorParams#clone method (Joep Rottinghuis); (stack: rev a5a48dbd23f3dab0f72635b2422f295a667b9ca4) * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutatorParams.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorParams.java > Add BufferedMutatorParams#clone method > -- > > Key: HBASE-17313 > URL: https://issues.apache.org/jira/browse/HBASE-17313 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Joep Rottinghuis >Assignee: Joep Rottinghuis > Fix For: 2.0.0 > > Attachments: HBASE-17313.master.001.patch, > HBASE-17313.master.002.patch > > > In HBASE-17277 we allow an alternate BufferedMutator implementation. If we > want to use that and not make BufferedMutatorImpl public, we need to be able > to ask the connection to create us one. In that case, we need to clone the > params (and then strip one argument off). > This jira is to add a clone method for convenience. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294, 'External Configuration for Memory Compaction'
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749370#comment-15749370 ] Hudson commented on HBASE-17316: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2132 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2132/]) HBASE-17316: Addendum of HBASE-17294 (stack: rev 691f266fc2f2b3d7bd9b3108f795f12b35e334c3) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java > Addendum to HBASE-17294, 'External Configuration for Memory Compaction' > --- > > Key: HBASE-17316 > URL: https://issues.apache.org/jira/browse/HBASE-17316 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 2.0.0 > > Attachments: HBASE-17316.patch > > > Updating 2 tests that failed during the commit of HBASE-17294 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees
[ https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749494#comment-15749494 ] huaxiang sun commented on HBASE-17309: -- Thanks Ted, will change and upload a new patch. > Fix connection leaks in TestAcidGuarantees > -- > > Key: HBASE-17309 > URL: https://issues.apache.org/jira/browse/HBASE-17309 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-17309-master-001.patch > > > Connections are not closed in some of TestAcidGuarantees's classes, this > makes integration test fail in some cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17149) Procedure v2 - Fix nonce submission
[
https://issues.apache.org/jira/browse/HBASE-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749539#comment-15749539
]
Hadoop QA commented on HBASE-17149:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 22 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 56s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 30s
{color} | {color:red} hbase-procedure generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 28s
{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed = 2
total (was 1) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 30s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 20s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 149m 12s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-procedure |
| | Boxed value is unboxed and then immediately reboxed in
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.setFailureResultForNonce(NonceKey,
String, User, IOException) At ProcedureExecutor.java:then immediately reboxed
in
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.setFailureResultForNonce(NonceKey,
String, User, IOException) At ProcedureExecutor.java:[line 745] |
| Failed junit tests |
hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843272/nonce.patch |
| JIRA Issue | HBASE-17149 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux e6bfa35f376d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x8
[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17292: --- Attachment: 17292.v2.txt Patch v2 addresses Vlad's comments > Add observer notification before bulk loaded hfile is moved to region > directory > --- > > Key: HBASE-17292 > URL: https://issues.apache.org/jira/browse/HBASE-17292 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 17292.v1.txt, 17292.v2.txt > > > Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded > hfiles. > However, if bulk load fails after hfile is moved to region directory but > before postBulkLoadHFile() hook is called, there is no way for pluggable > components (replication - see HBASE-17290, backup / restore) to know which > hfile(s) have been moved to region directory. > This issue adds a preCommitStoreFile() hook which notifies path of to be > committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16146) Counters are expensive...
[ https://issues.apache.org/jira/browse/HBASE-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749571#comment-15749571 ] stack commented on HBASE-16146: --- Ok I backport this one [~busbey]? > Counters are expensive... > - > > Key: HBASE-16146 > URL: https://issues.apache.org/jira/browse/HBASE-16146 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: Gary Helmling > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16146.001.patch, HBASE-16146.branch-1.001.patch, > HBASE-16146.branch-1.3.001.patch, counters.patch, less_and_less_counters.png > > > Doing workloadc, perf shows 10%+ of CPU being spent on counter#add. If I > disable some of the hot ones -- see patch -- I can get 10% more throughput > (390k to 440k). Figure something better. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17072) CPU usage starts to climb up to 90-100% when using G1GC; purge ThreadLocal usage
[
https://issues.apache.org/jira/browse/HBASE-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749575#comment-15749575
]
stack commented on HBASE-17072:
---
Ok to backport this [~busbey]
> CPU usage starts to climb up to 90-100% when using G1GC; purge ThreadLocal
> usage
>
>
> Key: HBASE-17072
> URL: https://issues.apache.org/jira/browse/HBASE-17072
> Project: HBase
> Issue Type: Bug
> Components: Performance, regionserver
>Affects Versions: 1.0.0, 2.0.0, 1.2.0
>Reporter: Eiichi Sato
>Assignee: Eiichi Sato
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 0.98.24
>
> Attachments: HBASE-17072-0.98.patch, HBASE-17072.branch-1.001.patch,
> HBASE-17072.master.001.patch, HBASE-17072.master.002.patch,
> HBASE-17072.master.003.patch, HBASE-17072.master.004.patch,
> HBASE-17072.master.005.patch, HBASE-17072.master.005.patch,
> disable-block-header-cache.patch, mat-threadlocals.png, mat-threads.png,
> metrics.png, slave1.svg, slave2.svg, slave3.svg, slave4.svg
>
>
> h5. Problem
> CPU usage of a region server in our CDH 5.4.5 cluster, at some point, starts
> to gradually get higher up to nearly 90-100% when using G1GC. We've also run
> into this problem on CDH 5.7.3 and CDH 5.8.2.
> In our production cluster, it normally takes a few weeks for this to happen
> after restarting a RS. We reproduced this on our test cluster and attached
> the results. Please note that, to make it easy to reproduce, we did some
> "anti-tuning" on a table when running tests.
> In metrics.png, soon after we started running some workloads against a test
> cluster (CDH 5.8.2) at about 7 p.m. CPU usage of the two RSs started to rise.
> Flame Graphs (slave1.svg to slave4.svg) are generated from jstack dumps of
> each RS process around 10:30 a.m. the next day.
> After investigating heapdumps from another occurrence on a test cluster
> running CDH 5.7.3, we found that the ThreadLocalMap contain a lot of
> contiguous entries of {{HFileBlock$PrefetchedHeader}} probably due to primary
> clustering. This caused more loops in
> {{ThreadLocalMap#expungeStaleEntries()}}, consuming a certain amount of CPU
> time. What is worse is that the method is called from RPC metrics code,
> which means even a small amount of per-RPC time soon adds up to a huge amount
> of CPU time.
> This is very similar to the issue in HBASE-16616, but we have many
> {{HFileBlock$PrefetchedHeader}} not only {{Counter$IndexHolder}} instances.
> Here are some OQL counts from Eclipse Memory Analyzer (MAT). This shows a
> number of ThreadLocal instances in the ThreadLocalMap of a single handler
> thread.
> {code}
> SELECT *
> FROM OBJECTS (SELECT AS RETAINED SET OBJECTS value
> FROM OBJECTS 0x4ee380430) obj
> WHERE obj.@clazz.@name =
> "org.apache.hadoop.hbase.io.hfile.HFileBlock$PrefetchedHeader"
> #=> 10980 instances
> {code}
> {code}
> SELECT *
> FROM OBJECTS (SELECT AS RETAINED SET OBJECTS value
> FROM OBJECTS 0x4ee380430) obj
> WHERE obj.@clazz.@name = "org.apache.hadoop.hbase.util.Counter$IndexHolder"
> #=> 2052 instances
> {code}
> Although as described in HBASE-16616 this somewhat seems to be an issue in
> G1GC side regarding weakly-reachable objects, we should keep ThreadLocal
> usage minimal and avoid creating an indefinite number (in this case, a number
> of HFiles) of ThreadLocal instances.
> HBASE-16146 removes ThreadLocals from the RPC metrics code. That may solve
> the issue (I just saw the patch, never tested it at all), but the
> {{HFileBlock$PrefetchedHeader}} are still there in the ThreadLocalMap, which
> may cause issues in the future again.
> h5. Our Solution
> We simply removed the whole {{HFileBlock$PrefetchedHeader}} caching and
> fortunately we didn't notice any performance degradation for our production
> workloads.
> Because the PrefetchedHeader caching uses ThreadLocal and because RPCs are
> handled randomly in any of the handlers, small Get or small Scan RPCs do not
> benefit from the caching (See HBASE-10676 and HBASE-11402 for the details).
> Probably, we need to see how well reads are saved by the caching for large
> Scan or Get RPCs and especially for compactions if we really remove the
> caching. It's probably better if we can remove ThreadLocals without breaking
> the current caching behavior.
> FWIW, I'm attaching the patch we applied. It's for CDH 5.4.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17292: --- Hadoop Flags: Reviewed Fix Version/s: 2.0.0 > Add observer notification before bulk loaded hfile is moved to region > directory > --- > > Key: HBASE-17292 > URL: https://issues.apache.org/jira/browse/HBASE-17292 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0 > > Attachments: 17292.v1.txt, 17292.v2.txt > > > Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded > hfiles. > However, if bulk load fails after hfile is moved to region directory but > before postBulkLoadHFile() hook is called, there is no way for pluggable > components (replication - see HBASE-17290, backup / restore) to know which > hfile(s) have been moved to region directory. > This issue adds a preCommitStoreFile() hook which notifies path of to be > committed hfile before bulk loaded hfile is moved to region directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749588#comment-15749588 ] stack commented on HBASE-15222: --- [~busbey] ok to backport to 1.2? > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, > HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, > HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, > HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, > HBASE-15222-v9.patch, HBASE-15222.patch > > > Running the benchmarks now, but it looks like the results are pretty extreme. > The locking in our histograms is pretty extreme. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17310) Refactoring BackupAdmin: remove public access modifiers
[ https://issues.apache.org/jira/browse/HBASE-17310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17310: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Integrated to HBASE-7912 branch > Refactoring BackupAdmin: remove public access modifiers > --- > > Key: HBASE-17310 > URL: https://issues.apache.org/jira/browse/HBASE-17310 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Trivial > Attachments: HBASE-17310.HBASE-7912.v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16146) Counters are expensive...
[ https://issues.apache.org/jira/browse/HBASE-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16146: Component/s: metrics > Counters are expensive... > - > > Key: HBASE-16146 > URL: https://issues.apache.org/jira/browse/HBASE-16146 > Project: HBase > Issue Type: Sub-task > Components: metrics >Reporter: stack >Assignee: Gary Helmling > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16146.001.patch, HBASE-16146.branch-1.001.patch, > HBASE-16146.branch-1.3.001.patch, counters.patch, less_and_less_counters.png > > > Doing workloadc, perf shows 10%+ of CPU being spent on counter#add. If I > disable some of the hot ones -- see patch -- I can get 10% more throughput > (390k to 440k). Figure something better. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-17308) BackupInfo getShortDescription output format
[ https://issues.apache.org/jira/browse/HBASE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-17308: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Integrated to HBASE-7912 branch. > BackupInfo getShortDescription output format > > > Key: HBASE-17308 > URL: https://issues.apache.org/jira/browse/HBASE-17308 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-17308.HBASE-7912.v1.patch > > > Currently its multiline and is hard to parse, convert it to a single line > output. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master
[
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749622#comment-15749622
]
Hadoop QA commented on HBASE-17000:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 4m 21s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
2s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
17s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
0s {color} | {color:green} HBASE-16961 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 50s
{color} | {color:red} hbase-protocol-shaded in HBASE-16961 has 24 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} HBASE-16961 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 11 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 2s
{color} | {color:red} The patch causes 270 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 4s
{color} | {color:red} The patch causes 270 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 6s
{color} | {color:red} The patch causes 270 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 7s
{color} | {color:red} The patch causes 270 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 9s
{color} | {color:red} The patch causes 270 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 23s
{color} | {color:red} Patch generated 1 new protoc errors in hbase-server.
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 87m 17s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 160m 49s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749628#comment-15749628 ] Sean Busbey commented on HBASE-15222: - this would be a backport for HBASE-15222, HBASE-12133, and HBASE-16146, right? what's the risk / downside? Have we gotten some operational feedback written down that shows metrics as a pain point? (I've heard through the grapevine a bit, but it's hard to rely on that as a matter of transparency.) > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, > HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, > HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, > HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, > HBASE-15222-v9.patch, HBASE-15222.patch > > > Running the benchmarks now, but it looks like the results are pretty extreme. > The locking in our histograms is pretty extreme. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master
[
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749668#comment-15749668
]
Josh Elser commented on HBASE-17000:
bq. The patch has 11 line(s) that end in whitespace. Use git apply
--whitespace=fix.
This is in generated code. Don't think it should be fixed.
bq. Patch generated 1 new protoc errors in hbase-server.
This is a false positive on the following line:
{{\[DEBUG\] adding entry
org/apache/hadoop/hbase/util/HBaseFsck$ErrorReporter$ERROR_CODE.class}}
bq. The patch causes 270 errors with Hadoop ...
These are expected as HBASE-16961 isn't treated like the master branch (which
it should be).
[~tedyu], shall I go ahead and commit this to HBASE-16961?
> [RegionServer] Compute region filesystem space use and report to Master
> ---
>
> Key: HBASE-17000
> URL: https://issues.apache.org/jira/browse/HBASE-17000
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17000.001.patch,
> HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch,
> HBASE-17000.005.HBASE-16961.patch
>
>
> Each RegionServer needs to track how much space a Region takes up and roll
> this up to the table level.
> Aggregation of this information in the Master will be covered by HBASE-16997.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-17307) Fix pom.xml (distcp version)
[
https://issues.apache.org/jira/browse/HBASE-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749669#comment-15749669
]
Ted Yu commented on HBASE-17307:
In hbase-server/pom.xml :
{code}
org.apache.hadoop
hadoop-distcp
${hadoop-two.version}
{code}
There is no need to add hadoop-two.version , right ?
I don't see such for the other neighboring dependencies.
> Fix pom.xml (distcp version)
>
>
> Key: HBASE-17307
> URL: https://issues.apache.org/jira/browse/HBASE-17307
> Project: HBase
> Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17307.HBASE-7912.v1.patch
>
>
> Currently, hbase-server/pom.xml has hardcoded version for distcp package,
> which is hadoop-two.version. This won't work for hadoop-three.version profile.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: HBase-16261-V7.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase, mapreduce >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, > HBase-16261-V6.patch, HBase-16261-V7.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master
[
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749698#comment-15749698
]
Ted Yu commented on HBASE-17000:
I ran the following locally:
{code}
Running org.apache.hadoop.hbase.quotas.TestRegionSizeUse
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.931 sec - in
org.apache.hadoop.hbase.quotas.TestRegionSizeUse
Running org.apache.hadoop.hbase.quotas.TestFileSystemUtilizationChore
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.443 sec - in
org.apache.hadoop.hbase.quotas.TestFileSystemUtilizationChore
Running
org.apache.hadoop.hbase.regionserver.TestRegionServerRegionSpaceUseReport
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.097 sec - in
org.apache.hadoop.hbase.regionserver.TestRegionServerRegionSpaceUseReport
{code}
Go ahead with commit.
> [RegionServer] Compute region filesystem space use and report to Master
> ---
>
> Key: HBASE-17000
> URL: https://issues.apache.org/jira/browse/HBASE-17000
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17000.001.patch,
> HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch,
> HBASE-17000.005.HBASE-16961.patch
>
>
> Each RegionServer needs to track how much space a Region takes up and roll
> this up to the table level.
> Aggregation of this information in the Master will be covered by HBASE-16997.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master
[ https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-17000: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review cycles, Ted and Stephen! Applied .005 to HBASE-16961. > [RegionServer] Compute region filesystem space use and report to Master > --- > > Key: HBASE-17000 > URL: https://issues.apache.org/jira/browse/HBASE-17000 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-17000.001.patch, > HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch, > HBASE-17000.005.HBASE-16961.patch > > > Each RegionServer needs to track how much space a Region takes up and roll > this up to the table level. > Aggregation of this information in the Master will be covered by HBASE-16997. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749737#comment-15749737 ] stack commented on HBASE-15222: --- Not sure about risk/downside. It is a big patch. Maybe too risky then [~busbey]. The original thread dump here looks to be clearing of thread local reference q. Maybe letting in the two thread local changes would be enough late in 1.2 cycle. > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, > HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, > HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, > HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, > HBASE-15222-v9.patch, HBASE-15222.patch > > > Running the benchmarks now, but it looks like the results are pretty extreme. > The locking in our histograms is pretty extreme. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15222) Use less contended classes for metrics
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749760#comment-15749760 ] stack commented on HBASE-15222: --- [~ghelmling] You fellows running this in prod? > Use less contended classes for metrics > -- > > Key: HBASE-15222 > URL: https://issues.apache.org/jira/browse/HBASE-15222 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, > HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, > HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, > HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, > HBASE-15222-v9.patch, HBASE-15222.patch > > > Running the benchmarks now, but it looks like the results are pretty extreme. > The locking in our histograms is pretty extreme. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: (was: HBase-16261-V7.patch) > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase, mapreduce >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, > HBase-16261-V6.patch, HBase-16261-V7.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: HBase-16261-V7.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase, mapreduce >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, > HBase-16261-V6.patch, HBase-16261-V7.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16998) [Master] Analyze table use reports and update quota violations
[ https://issues.apache.org/jira/browse/HBASE-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-16998: --- Attachment: HBASE-16998.002.HBASE-16961.patch .002 is .001 rebased on top of HBASE-16961. > [Master] Analyze table use reports and update quota violations > -- > > Key: HBASE-16998 > URL: https://issues.apache.org/jira/browse/HBASE-16998 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 2.0.0 > > Attachments: HBASE-16998.001.patch, HBASE-16998.002.HBASE-16961.patch > > > Given the collected table usage reports from RegionServers, the Master needs > to inspect all filesystem-use quotas and determine which need to move into > violation and which need to move out of violation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16998) [Master] Analyze table use reports and update quota violations
[
https://issues.apache.org/jira/browse/HBASE-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749777#comment-15749777
]
Hadoop QA commented on HBASE-16998:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 14s
{color} | {color:red} Docker failed to build yetus/hbase:8d52d23. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843313/HBASE-16998.002.HBASE-16961.patch
|
| JIRA Issue | HBASE-16998 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4922/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [Master] Analyze table use reports and update quota violations
> --
>
> Key: HBASE-16998
> URL: https://issues.apache.org/jira/browse/HBASE-16998
> Project: HBase
> Issue Type: Sub-task
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-16998.001.patch, HBASE-16998.002.HBASE-16961.patch
>
>
> Given the collected table usage reports from RegionServers, the Master needs
> to inspect all filesystem-use quotas and determine which need to move into
> violation and which need to move out of violation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-17309) Fix connection leaks in TestAcidGuarantees
[ https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-17309: - Attachment: HBASE-17309-master-002.patch > Fix connection leaks in TestAcidGuarantees > -- > > Key: HBASE-17309 > URL: https://issues.apache.org/jira/browse/HBASE-17309 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-17309-master-001.patch, > HBASE-17309-master-002.patch > > > Connections are not closed in some of TestAcidGuarantees's classes, this > makes integration test fail in some cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees
[ https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749843#comment-15749843 ] huaxiang sun commented on HBASE-17309: -- New patch to address Ted's comments. > Fix connection leaks in TestAcidGuarantees > -- > > Key: HBASE-17309 > URL: https://issues.apache.org/jira/browse/HBASE-17309 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-17309-master-001.patch, > HBASE-17309-master-002.patch > > > Connections are not closed in some of TestAcidGuarantees's classes, this > makes integration test fail in some cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
[
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749848#comment-15749848
]
Hadoop QA commented on HBASE-17292:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
23m 11s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 87m 1s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 121m 1s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843297/17292.v2.txt |
| JIRA Issue | HBASE-17292 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 1809d5831c98 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 691f266 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4919/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4919/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add observer notification before bulk loaded hfile is moved to region
> directory
[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees
[ https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749849#comment-15749849 ] Ted Yu commented on HBASE-17309: +1, if test passes. > Fix connection leaks in TestAcidGuarantees > -- > > Key: HBASE-17309 > URL: https://issues.apache.org/jira/browse/HBASE-17309 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 2.0.0 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-17309-master-001.patch, > HBASE-17309-master-002.patch > > > Connections are not closed in some of TestAcidGuarantees's classes, this > makes integration test fail in some cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-17311) Refactoring: move backup system table out of 'hbase' namespace
[ https://issues.apache.org/jira/browse/HBASE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749911#comment-15749911 ] Ted Yu commented on HBASE-17311: When 'BACKUP.SYSTEM' table exists at cluster startup, backup feature needs to be turned off and ERROR should be logged, asking user to configure a different table as backup table. > Refactoring: move backup system table out of 'hbase' namespace > -- > > Key: HBASE-17311 > URL: https://issues.apache.org/jira/browse/HBASE-17311 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-17311.HBASE-7912.v1.patch > > > In preparation to big refactoring (separate project for backup) we need to > move 'hbase:system' table out of HBase system namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement
[
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749948#comment-15749948
]
Hadoop QA commented on HBASE-16261:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m 19s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 44s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843307/HBase-16261-V7.patch |
| JIRA Issue | HBASE-16261 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c241cd105f23 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 691f266 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4920/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4920/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> MultiHFileOutputFormat Enhancement
>
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
> Issue Type: Sub-task
> Components: hbase, mapreduce
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-162
[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement
[
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749998#comment-15749998
]
Hadoop QA commented on HBASE-16261:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 40s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 24s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 23s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 129m 10s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12843312/HBase-16261-V7.patch |
| JIRA Issue | HBASE-16261 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux a6fa3167d022 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 691f266 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/4921/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/4921/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> MultiHFileOutputFormat Enhancement
>
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
> Issue Type: Sub-task
> Components: hbase, mapreduce
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-1626
[jira] [Updated] (HBASE-17296) Provide per peer throttling for replication
[ https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-17296: --- Attachment: (was: HBASE-17296-branch-1.patch) > Provide per peer throttling for replication > --- > > Key: HBASE-17296 > URL: https://issues.apache.org/jira/browse/HBASE-17296 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-17296-branch-1.patch, HBASE-17296.patch > > > HBASE-9501 added a config to provide throttling for replication. And each > peer has same bandwidth up limit. In our use case, one cluster may have > several peers and several slave clusters. Each slave cluster may have > different scales and need different bandwidth up limit for each peer. So We > add bandwidth to replication peer config and provide a shell cmd set_peer > bandwidth to update the bandwidth in need. It has been used for a long time > on our clusters. Any suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
