Strange folder inside WALs

I see a folder named hregion-47776456 inside /hbase/WALs folder. After a bit of digging, I see its being created by createRegion function in HRegion. What is the use of this folder? There is no mention of this folder in http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html. (Admit

Re: How to Manage Data Architecture & Modeling for HBase

Yeah. Jean-Marc is right. You have to think more in terms of a hierarchical model where you’re modeling records not relationships. Your model would look like a single ER box per record type. The HBase schema is very simple. Tables, column families and that’s it for static structures. Even

Re: How to Manage Data Architecture & Modeling for HBase

> tools to manage Data Architecture & Modeling for HBase To aid visualizing table structure, you could use Enterprise Architect Even though HBase cells store BLOBS, quite often these BLOBS are serialized classes. In EA classes can appear in table definition as field types. It is possible to pu

Re: How to Manage Data Architecture & Modeling for HBase

Thank you for your prompt reply. In my daily work, I mainly used Oracle DB to build a data warehouse with star topology data modeling, about financial analysis and marketing analysis. Now I trying to use Hbase to do it. I has a question, 1) many tables from ERP should be Incremental loading ev

Strange PrefixFilter behaviour on HBase 0.98.6-cdh5.2.0 OutOfOrderScannerNextException

Hi, I'm trying to use PrefixFilter for the RowKey. My rowKey consists of 3 parts, actually it's composite. I do provide first part of key to scan all rows starting from prefix. There should be less than 10 rowkeys for each prefix, since prefix is md5 hash. I have itests for this part of code, it ru

Re: Strange PrefixFilter behaviour on HBase 0.98.6-cdh5.2.0 OutOfOrderScannerNextException

Looks like I didn't set startRow for the scanner... 2015-04-06 17:04 GMT+02:00 Serega Sheypak : > Hi, I'm trying to use PrefixFilter for the RowKey. > My rowKey consists of 3 parts, actually it's composite. > I do provide first part of key to scan all rows starting from prefix. > There should be

Re: Strange PrefixFilter behaviour on HBase 0.98.6-cdh5.2.0 OutOfOrderScannerNextException

may this be related: https://issues.apache.org/jira/browse/HBASE-11295 ?

Re: How to Manage Data Architecture & Modeling for HBase

So this is the hardest thing to do… teach someone not to look at the data in terms of an RDBMs model. And there aren’t any hard and fast rules… Lets look at an example. You’re creating an application for Medicare/Medicaid to help identify potential abuses and fraud within the system. In p

Re: How to Manage Data Architecture & Modeling for HBase

I should add that in terms of financial modeling… Its easier to store derivatives and synthetic instruments because you aren’t really constrained by a relational model. (Derivatives are nothing more than a contract.) HTH -Mike > On Apr 6, 2015, at 8:34 AM, Ben Liang wrote: > > Thank you f

RE: How to Manage Data Architecture & Modeling for HBase

I would stress that if you envision any joins or arbitrary slices and dices at a later point in your application, you might want to either redesign your schema "very carefully" or be ready for more time consuming ( not near real time) answers. We had explored a possible solution on similar line

Re: Please welcome new HBase committer Srikanth Srungarapu

Nice work Srikanth! On Thu, Apr 2, 2015 at 10:49 PM, Mikhail Antonov wrote: > Congratulations Srikanth! > > -Mikhail > > On Thu, Apr 2, 2015 at 8:44 PM, Pankaj kr wrote: > > Congrats Srikanth..!! > > > > -Original Message- > > From: Andrew Purtell [mailto:apurt...@apache.org] > > Sent:

Re: Please welcome new HBase committer Jing Chen (Jerry) He

Thanks a lot for the efforts Jerry, welcome. On Fri, Apr 3, 2015 at 1:46 AM, Srikanth Srungarapu wrote: > Kudos, Jerry! > > On Thu, Apr 2, 2015 at 10:49 PM, Mikhail Antonov > wrote: > > > Congrats Jerry! > > > > -Mikhail > > > > On Thu, Apr 2, 2015 at 9:21 PM, Pankaj kr wrote: > > > Congrats J

Where is up to date documentation and dependencies for HBaseTestingUtility class?

I can't find the Javadoc for HBaseTestingUtility. The maven dependencies shown in Section 140 of the HBase Book are also wrong. Maven can't resolve them: http://hbase.apache.org/book.html#_integration_testing_with_a_hbase_mini_cluster Where are the Javadocs and up-to-date dependencies for

Re: Strange PrefixFilter behaviour on HBase 0.98.6-cdh5.2.0 OutOfOrderScannerNextException

I forgot to set firstRow for Scanner. Looks like HBase tried to scan the whole table. Value from FilterPrefix wasn't used. I supposed that prefix value could be pushed to scanner as a starting point, but not. 2015-04-06 18:45 GMT+02:00 Imants Cekusins : > may this be related: > > https://issues.

Re: Strange PrefixFilter behaviour on HBase 0.98.6-cdh5.2.0 OutOfOrderScannerNextException

>Yes, scan goes through entire table unless start row is set. > does this explain the error though? > Prefix filter should work even with scan beginning from 1st record, no? It would only take longer. Yes, it's explains. My table has 70M rows, and prefix filter should scan just for 10 starting

Monitoring tools for Hbase

Hi, Could someone recommend the good monitoring tools for Hbase to see distribution of data over the regions, Hfiles for a table, how the reads are happening on each region and to collect some metrics. Any recommendations are appreciated. Thanks, Siva.

Re: Strange PrefixFilter behaviour on HBase 0.98.6-cdh5.2.0 OutOfOrderScannerNextException

Hi Serega, Yes, you need to set start row when using Prefix filter. Another smart trick for prefix filter: Set the prefix as start-row of scan and set the next byteorder successor(next bigger byte array with same number of bytes) as stop row of your scan. For example(using chars for simplicity):

Re: Monitoring tools for Hbase

Cloudera Manager if you are using a distributor like them. If not, OpenTSDB together with Grafana allows you to graph anything and annotate it which may be useful when you deploy code that may cause interesting behavior. On Mon, Apr 6, 2015 at 5:33 PM, Siva wrote: > Hi, > > Could someone recomme

Re: Where is up to date documentation and dependencies for HBaseTestingUtility class?

You can run 'mvn dependency:tree' (with proper defines such as -Dhadoop-two.version=x.y) in the root of workspace of the hbase release you're using. This would generate dependencies you were looking for. Cheers On Mon, Apr 6, 2015 at 1:21 PM, Schulze, Dean (Contractor) < dean_schu...@cable.comca

Re: Monitoring tools for Hbase

Hi Siva, Have a look at SPM . It captures a pile of HBase metrics , has built in alerting, anomaly detection, event and log correlation, even transaction tracking. Here's one of the interesting HBase metric