Re: Mass dumping of data has issues

2012-09-24 Thread Paul Mackles
Did you adjust the writebuffer to a larger size and/or turn off autoFlush for the Htable? I've found that both of those settings can have a profound impact on write performance. You might also look at adjusting the handler count for the regionservers which by default is pretty low. You should also

Re: Mass dumping of data has issues

2012-09-25 Thread Dan Han
Hi Naveen, There is tool called Sqoop which supports importing the data from relational database to HBase. https://blogs.apache.org/sqoop/entry/apache_sqoop_graduates_from_incubator Maybe it can help you migrate the data easily. Best Wishes Dan Han On Mon, Sep 24, 2012 at 9:20 AM, Paul Mackl

RE: Mass dumping of data has issues

2012-09-25 Thread Naveen
n -Original Message- From: Paul Mackles [mailto:pmack...@adobe.com] Sent: Monday, September 24, 2012 8:51 PM To: user@hbase.apache.org Subject: Re: Mass dumping of data has issues Did you adjust the writebuffer to a larger size and/or turn off autoFlush for the Htable? I've found th

RE: Mass dumping of data has issues

2012-09-25 Thread Naveen
ailto:dannahan2...@gmail.com] Sent: Wednesday, September 26, 2012 7:20 AM To: user@hbase.apache.org Subject: Re: Mass dumping of data has issues Hi Naveen, There is tool called Sqoop which supports importing the data from relational database to HBase. https://blogs.apache.

RE: Mass dumping of data has issues

2012-09-25 Thread Ramkrishna.S.Vasudevan
Naveen [mailto:naveen.moorj...@cleartrip.com] > Sent: Wednesday, September 26, 2012 11:52 AM > To: user@hbase.apache.org > Subject: RE: Mass dumping of data has issues > > Hi Dan, > > I'm actually trying to simulate the kind of load we're expecting on > production servers(the in

RE: Mass dumping of data has issues

2012-09-26 Thread Naveen
2:15 PM To: user@hbase.apache.org Subject: RE: Mass dumping of data has issues For the NPE that you got, is the same HTable instance shared by different threads. This is a common problem users encounter while using HTable across multiple threads. Pls check and ensure that the HTable is not shared