[jira] [Created] (HBASE-18910) Backport HBASE-17292 "Add observer notification before bulk loaded hfile is moved to region directory" to 1.3

Guangxu Cheng created HBASE-18910: - Summary: Backport HBASE-17292 "Add observer notification before bulk loaded hfile is moved to region directory" to 1.3 Key: HBASE-18910 URL:

[jira] [Created] (HBASE-18909) Deprecate Admin's methods which used String regex

Guanghao Zhang created HBASE-18909: -- Summary: Deprecate Admin's methods which used String regex Key: HBASE-18909 URL: https://issues.apache.org/jira/browse/HBASE-18909 Project: HBase Issue

[jira] [Created] (HBASE-18908) Add Java 9 section to support matrix documentation

Mike Drob created HBASE-18908: - Summary: Add Java 9 section to support matrix documentation Key: HBASE-18908 URL: https://issues.apache.org/jira/browse/HBASE-18908 Project: HBase Issue Type:

Re: Performance issue in the Join query on the HBase tables

@Eric: for the trafodion, will take a look. @Nick: And for the Hive/Spark over snapshots, I just have a try on the Hive over HBase snapshots, the select(count) is much more faster than Hive over HBase. Since the HBase tables are all so big, how to make the engine respecting the data locality?

Re: Welcome Chia-Ping Tsai to the HBase PMC

Well deserved, Chia-Ping! On Fri, Sep 29, 2017 at 6:04 PM, Esteban Gutierrez wrote: > Congrats Chia-Ping! and Welcome! > > -- > Cloudera, Inc. > > > On Fri, Sep 29, 2017 at 3:52 PM, Guanghao Zhang > wrote: > > > Congratulations! > > > > 2017-09-30

[jira] [Resolved] (HBASE-18559) Add histogram to MetricsConnection to track concurrent calls per server

[ https://issues.apache.org/jira/browse/HBASE-18559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-18559. Resolution: Fixed Hadoop Flags: Reviewed Pushed to 1.4 and up > Add histogram to

Re: Welcome Chia-Ping Tsai to the HBase PMC

Welcome Chia-Ping. Keep up the great work. S On Fri, Sep 29, 2017 at 3:19 PM, Misty Stanley-Jones wrote: > The HBase PMC is delighted to announce that Chia-Ping Tsai has agreed to > join > the HBase PMC, and help to make the project run smoothly. Chia-Ping became > an > HBase

[jira] [Resolved] (HBASE-18436) Add client-side hedged read metrics

[ https://issues.apache.org/jira/browse/HBASE-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-18436. Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.5.0

Re: Welcome Chia-Ping Tsai to the HBase PMC

Congrats Chia-Ping! and Welcome! -- Cloudera, Inc. On Fri, Sep 29, 2017 at 3:52 PM, Guanghao Zhang wrote: > Congratulations! > > 2017-09-30 6:38 GMT+08:00 Andrew Purtell : > > > Congratulations, Chia-Ping! Welcome to the PMC. > > > > On Fri, Sep 29,

Re: Welcome Chia-Ping Tsai to the HBase PMC

Congratulations Chia-Ping! Huaxiang > On Sep 29, 2017, at 3:52 PM, Guanghao Zhang wrote: > > Congratulations! > > 2017-09-30 6:38 GMT+08:00 Andrew Purtell : > >> Congratulations, Chia-Ping! Welcome to the PMC. >> >> On Fri, Sep 29, 2017 at 3:19 PM,

Re: Welcome Chia-Ping Tsai to the HBase PMC

Congratulations! 2017-09-30 6:38 GMT+08:00 Andrew Purtell : > Congratulations, Chia-Ping! Welcome to the PMC. > > On Fri, Sep 29, 2017 at 3:19 PM, Misty Stanley-Jones > wrote: > > > The HBase PMC is delighted to announce that Chia-Ping Tsai has agreed to >

[jira] [Created] (HBASE-18907) Methods missing rpc timeout parameter in HTable

Ted Yu created HBASE-18907: -- Summary: Methods missing rpc timeout parameter in HTable Key: HBASE-18907 URL: https://issues.apache.org/jira/browse/HBASE-18907 Project: HBase Issue Type: Bug

Re: Welcome Chia-Ping Tsai to the HBase PMC

Congratulations, Chia-Ping! Welcome to the PMC. On Fri, Sep 29, 2017 at 3:19 PM, Misty Stanley-Jones wrote: > The HBase PMC is delighted to announce that Chia-Ping Tsai has agreed to > join > the HBase PMC, and help to make the project run smoothly. Chia-Ping became > an >

Re: Welcome Chia-Ping Tsai to the HBase PMC

My sincere congratulations! On Fri, Sep 29, 2017 at 3:22 PM, Ted Yu wrote: > Congratulations, Chia-Ping. > > On Fri, Sep 29, 2017 at 3:19 PM, Misty Stanley-Jones > wrote: > > > The HBase PMC is delighted to announce that Chia-Ping Tsai has agreed to > >

Re: Welcome Chia-Ping Tsai to the HBase PMC

Congratulations, Chia-Ping. On Fri, Sep 29, 2017 at 3:19 PM, Misty Stanley-Jones wrote: > The HBase PMC is delighted to announce that Chia-Ping Tsai has agreed to > join > the HBase PMC, and help to make the project run smoothly. Chia-Ping became > an > HBase committer over 6

Welcome Chia-Ping Tsai to the HBase PMC

The HBase PMC is delighted to announce that Chia-Ping Tsai has agreed to join the HBase PMC, and help to make the project run smoothly. Chia-Ping became an HBase committer over 6 months ago, based on long-running participate in the HBase project, a consistent record of resolving HBase issues, and

Re: [DISCUSS] Move Type out of KeyValue

​Construct a normal put or delete or batch mutation, add whatever extra state you need in one or more operation attributes, and use a regionobserver to extend normal processing to handle the extra state. I'm curious what dispatching to extension code because of a custom cell type buys you over

Re: [DISCUSS] Move Type out of KeyValue

> Instead of a custom cell, could you use a regular cell with a custom > operation attribute (see OperationWithAttributes). Pardon me, I didn't get what you said. On 2017-09-30 04:31, Andrew Purtell wrote: > Instead of a custom cell, could you use a regular cell with a

Re: [DISCUSS] Move Type out of KeyValue

Instead of a custom cell, could you use a regular cell with a custom operation attribute (see OperationWithAttributes). On Fri, Sep 29, 2017 at 1:28 PM, Chia-Ping Tsai wrote: > The custom cell help us to save memory consumption. We don't have own >

Re: [DISCUSS] Move Type out of KeyValue

The custom cell help us to save memory consumption. We don't have own serialization/deserialization mechanism, hence to transform data from client to server needs many conversion phase (user data -> Put/Cell -> pb object). The cost of conversion is large in transferring bulk data. In fact, we

[DISCUSS] deprecating o.a.h.h.regionserver.RowProcessor

Hi, Currently Region.processRowsWithLocks() API takes o.a.h.h.regionserver.RowProcessor as an argument and only implementation of this class is MultiRowMutationProcessor. This implementation is internal and used from HRegion.mutateRows...() methods. HRegion.processRowsWithLocks() implementation,

Re: [DISCUSS] Move Type out of KeyValue

What are the use cases for a custom cell? It seems a dangerously low level thing to attempt and perhaps we should unwind support for it. But perhaps there is a compelling justification. On Thu, Sep 28, 2017 at 10:20 PM, Chia-Ping Tsai wrote: > Thanks for all comment. > >

Re: [DISCUSS] Becoming a Committer

This conversation is in a good place. I apologize for the tone of my earlier allergic reaction but not the content. I hope that is acceptable. On Fri, Sep 29, 2017 at 11:01 AM, Mike Drob wrote: > To bounce off of what Yu Li said earlier - I see Hadoop has adopted very >

Re: [DISCUSS] Becoming a Committer

To bounce off of what Yu Li said earlier - I see Hadoop has adopted very similar language to the Spark list: http://hadoop.apache.org/committer_criteria.html I especially like the examples at the bottom. They are four diverse paths, and there is no expectation that this is an exclusive list. If

Re: Performance issue in the Join query on the HBase tables

Have you considered running Hive/Spark over snapshots of your HBase tables? If you're seeing network saturation over HBase but not hdfs, makes me think data locality is not being honored. Might be worth investigating as well. On Fri, Sep 29, 2017 at 3:26 AM wenxing zheng

RE: Performance issue in the Join query on the HBase tables

Hi Wenxing, From the use case you describe, you may want to take a look at Trafodion or EsgynDB (commercial version of Trafodion). http://trafodion.incubator.apache.org/ Trafodion uses a very mature SQL engine on top of HBASE/HIVE coming with 20 years of IP given away to open source by

Re: Performance issue in the Join query on the HBase tables

Thanks to Ted. We didn't try the phoneix yet. From the performance test on the official site of phoenix, I didn't find the report on the Join query. Not sure whether it's much better or not On Fri, Sep 29, 2017 at 8:01 PM, Ted Yu wrote: > Have you looked at Phoenix ? > >

Re: Performance issue in the Join query on the HBase tables

Have you looked at Phoenix ? https://phoenix.apache.org/joins.html On Fri, Sep 29, 2017 at 3:25 AM, wenxing zheng wrote: > Dear all, > > I have 3 big HBase tables, which all have millions of rows(rows are synced > from MySQL DB via Bin log) and for each HBase table, we

Re: [DISCUSS] Move Type out of KeyValue

Ya as Chia-Ping said, the problem he is trying to solve is very basic one. As long as we allow custom Cell creation (Via CellBuilder API) and allow Mutations to be added with Cells and pass that from client side APIs, we have to make the Type public accessible. Or else the Cell building APIs

Performance issue in the Join query on the HBase tables

Dear all, I have 3 big HBase tables, which all have millions of rows(rows are synced from MySQL DB via Bin log) and for each HBase table, we have an external table on Hive correspondingly with the storage by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'. The advantage is that we can always

Re: [DISCUSS] Move Type out of KeyValue

Even if we are trying to move out I think only few of the types are really user readable. So we should be very careful here. So since we have CellBuilder way it is better we check what type of cells a user can build. I think for now the Cellbuilder is not client exposed? But again moving to Cell