Insert a lot of data in HBase

2011-06-20 Thread Laurent Hatier
Hi all, I'm new in HBase. I want to insert 4'000'000 rows in HBase (each row has 4 columns). I have already looked the HBase wiki to insert data, but i've a problem : i loss data. When i do a COUNT with the shell, there is approximativly 1'500'000 in the DB... I've tested to create multiple Put an

Re: Insert a lot of data in HBase

2011-06-20 Thread Jean-Daniel Cryans
4M is small data :) Could there be an overlap in the keys? Are you disabling autoflush and not flushing the write buffer? These are common errors/misconceptions. J-D On Mon, Jun 20, 2011 at 10:36 AM, Laurent Hatier wrote: > Hi all, > > I'm new in HBase. I want to insert 4'000'000 rows in HBase

RE: Insert a lot of data in HBase

2011-06-20 Thread Doug Meil
: Insert a lot of data in HBase 4M is small data :) Could there be an overlap in the keys? Are you disabling autoflush and not flushing the write buffer? These are common errors/misconceptions. J-D On Mon, Jun 20, 2011 at 10:36 AM, Laurent Hatier wrote: > Hi all, > > I'm new in HB

Re: Insert a lot of data in HBase

2011-06-20 Thread Laurent Hatier
t; > http://hbase.apache.org/book.html#performance > > > -Original Message- > From: jdcry...@gmail.com [mailto:jdcry...@gmail.com] On Behalf Of > Jean-Daniel Cryans > Sent: Monday, June 20, 2011 2:03 PM > To: user@hbase.apache.org > Subject: Re: Insert a lot of data in HBa

Re: Insert a lot of data in HBase

2011-06-20 Thread Laurent Hatier
mail.com] On Behalf Of >> Jean-Daniel Cryans >> Sent: Monday, June 20, 2011 2:03 PM >> To: user@hbase.apache.org >> Subject: Re: Insert a lot of data in HBase >> >> 4M is small data :) >> >> Could there be an overlap in the keys? Are you disabling autoflu

Re: Insert a lot of data in HBase

2011-06-20 Thread Jeff Whiting
riginal Message- From: jdcry...@gmail.com [mailto:jdcry...@gmail.com] On Behalf Of Jean-Daniel Cryans Sent: Monday, June 20, 2011 2:03 PM To: user@hbase.apache.org Subject: Re: Insert a lot of data in HBase 4M is small data :) Could there be an overlap in the keys? Are you disabling autoflush and n

Re: Insert a lot of data in HBase

2011-06-20 Thread Sam Seigal
;>> >>> Look here in the HBase book for these, and other, tips. >>>> >>>> http://hbase.apache.org/book.**html#performance<http://hbase.apache.org/book.html#performance> >>>> >>>> >>>> -Original Message- >>>> From:

RE: Insert a lot of data in HBase

2011-06-20 Thread Doug Meil
eigal Sent: Monday, June 20, 2011 10:17 PM To: user@hbase.apache.org Subject: Re: Insert a lot of data in HBase When using the write cache and setting setAutoFlush() to false, is there a risk of data loss, even if WAL is enabled ? On Mon, Jun 20, 2011 at 12:27 PM, Jeff Whiting wrote: >