I would stress that if you envision any joins or arbitrary slices and dices at a later point in your application, you might want to either redesign your schema "very carefully" or be ready for more time consuming ( not near real time) answers. We had explored a possible solution on similar lines but a hashtable approach (as expected) isn’t the best for database joins OR slicing based on arbitrary columns across the whole dataset. We had to switch back to a relational db for our usecase.
Thanks, Abhishek -----Original Message----- From: Michael Segel [mailto:michael_se...@hotmail.com] Sent: Monday, April 06, 2015 9:55 AM To: user@hbase.apache.org Cc: u...@phoenix.apache.org Subject: Re: How to Manage Data Architecture & Modeling for HBase I should add that in terms of financial modeling… Its easier to store derivatives and synthetic instruments because you aren’t really constrained by a relational model. (Derivatives are nothing more than a contract.) HTH -Mike > On Apr 6, 2015, at 8:34 AM, Ben Liang <lian...@hotmail.com> wrote: > > Thank you for your prompt reply. > > In my daily work, I mainly used Oracle DB to build a data warehouse with star > topology data modeling, about financial analysis and marketing analysis. > Now I trying to use Hbase to do it. > > I has a question, > 1) many tables from ERP should be Incremental loading every day , > Including some insert and some update, this scenario is appropriate > to use hbase to build data worehose？ > 2) Is there some case about Enterprise BI Solutions with HBASE? > > thanks. > > > Regards, > Ben Liang > >> On Apr 6, 2015, at 20:27, Michael Segel <michael_se...@hotmail.com> wrote: >> >> Yeah. Jean-Marc is right. >> >> You have to think more in terms of a hierarchical model where you’re >> modeling records not relationships. >> >> Your model would look like a single ER box per record type. >> >> The HBase schema is very simple. Tables, column families and that’s it for >> static structures. Even then, column families tend to get misused. >> >> If you’re looking at a relational model… Phoenix or Splice Machines would >> allow you to do something… although Phoenix is still VERY primitive. >> (Do they take advantage of cell versioning like spice machines yet? ) >> >> >> There are a couple of interesting things where you could create your >> own modeling tool / syntax (relationships)… >> >> 1) HBase is more 3D than RDBMS 2D and similar to ORDBMSs. >> 2) You can join entities on either a FK principle or on a weaker >> relationship type. >> >> HBase stores CLOBS/BLOBs in each cell. Its all just byte arrays with a >> finite bounded length not to exceed the size of a region. So you could store >> an entire record as a CLOB within a cell. Its in this sense that a cell can >> represent multiple attributes of your object/record that you gain an >> additional dimension and why you only need to use a single data type. >> >> HBase and Hadoop in general allow one to join orthogonal data sets that have >> a weak relationship. So while you can still join sets against a FK which >> implies a relationship, you don’t have to do it. >> >> Imagine if you wanted to find out the average cost of a front end collision >> by car of college aged drivers by major. >> You would be joining insurance records against registrations for all of the >> universities in the US for those students between the ages of 17 and 25. >> >> How would you model this when in fact neither defining attribute is a FK? >> (This is why you need a good Secondary Indexing implementation and >> not something brain dead that wasn’t alcohol induced. ;-) >> >> Does that make sense? >> >> Note: I don’t know if anyone like CCCis, Allstate, State Farm, or >> Progressive Insurance are doing anything like this. But they could. >> >>> On Apr 5, 2015, at 7:54 PM, Jean-Marc Spaggiari <jean-m...@spaggiari.org> >>> wrote: >>> >>> Not sure you want to ever do that... Designing an HBase application >>> is far different from designing an RDBMS one. Not sure those tools fit well >>> here. >>> >>> What's you're goal? Designing your HBase schema somewhere and then >>> let the tool generate your HBase tables? >>> >>> 2015-04-05 18:26 GMT-04:00 Ben Liang <lian...@hotmail.com>: >>> >>>> Hi all, >>>> Do you have any tools to manage Data Architecture & Modeling >>>> for HBase( or Phoenix) ? Can we use Powerdesinger or ERWin to do it? >>>> >>>> Please give me some advice. >>>> >>>> Regards, >>>> Ben Liang >>>> >>>> >> >> The opinions expressed here are mine, while they may reflect a cognitive >> thought, that is purely accidental. >> Use at your own risk. >> Michael Segel >> michael_segel (AT) hotmail.com >> >> >> >> >> > The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental. Use at your own risk. Michael Segel michael_segel (AT) hotmail.com
