Kranthi, Your tables seem to be small. Why do you want to port them to HBase?
-Amandeep Amandeep Khurana Computer Science Graduate Student University of California, Santa Cruz On Mon, Apr 12, 2010 at 1:55 AM, kranthi reddy <kranthili2...@gmail.com>wrote: > HI jonathan, > > Sorry for the late response. Missed your reply. > > The problem is, around 80% (400) of the tables are static tables and the > remaining 20% (100) are dynamic tables that are updated on a daily basis. > The problem is denormalising these 20% tables is also extremely difficult > and we are planning to port them directly into hbase. And also > denormalising > these tables would lead to a lot of redundant data. > > Static tables have number of entries varying in hundreds and mostly less > than 1000 entries (rows). Where as the dynamic tables have more than 20,000 > entries and each entry might be updated/modified at least once in a week. > > Regards, > kranthi > > > On Wed, Mar 31, 2010 at 10:23 PM, Jonathan Gray <jg...@facebook.com> > wrote: > > > Kranthi, > > > > HBase can handle a good number of tables, but tens or maybe a hundred. > If > > you have 500 tables you should definitely be rethinking your schema > design. > > The issue is less about HBase being able to handle lots of tables, and > much > > more about whether scattering your data across lots of tables will be > > performant at read time. > > > > > > 1) Impossible to answer that question without knowing the schemas of the > > existing tables. > > > > 2) Not really any relation between fault tolerance and the number of > > tables except potentially for recovery time but this would be the same > with > > few, very large tables. > > > > 3) No difference in write performance. Read performance if doing simple > > key lookups would not be impacted, but most like having data spread out > like > > this will mean you'll need joins of some sort. > > > > Can you tell more about your data and queries? > > > > JG > > > > > -----Original Message----- > > > From: kranthi reddy [mailto:kranthili2...@gmail.com] > > > Sent: Wednesday, March 31, 2010 3:05 AM > > > To: hbase-user@hadoop.apache.org > > > Subject: Porting SQL DB into HBASE > > > > > > Hi all, > > > > > > I have run into some trouble while trying to port SQL DB to > > > Hbase. > > > The problem is my SQL DB has around 500 tables (approx) and it is very > > > badly > > > designed. Around 45-50 tables could be denormalised into a single table > > > and > > > the remaining tables are static tables. My doubts are > > > > > > 1) Is it possible to port this DB (Tables) to Hbase? If possible how? > > > 2) How many tables can Hbase support with tolerance towards failure? > > > 3) When so many tables are inserted, how is the performance going to be > > > effected? Will it remain same or degrade? > > > > > > One possible solution I think is using column family for each table. > > > But as > > > per my knowledge and previous experiments, I found Hbase isn't stable > > > when > > > column families are more than 5. > > > > > > Since every day large quantities of data is ported into the DataBase, > > > stability and fail proof system is highest priority. > > > > > > Hoping for a positive response. > > > > > > Thank you, > > > kranthi > > > > > > -- > Kranthi Reddy. B > Room No : 98 > Old Boys Hostel > IIIT-HYD > > ----------- > > I don't know the key to success, but the key to failure is trying to > impress > others. >
