Re: regarding datamodel inside HBase

2009-01-13 Thread Jean-Daniel Cryans
Shiraz, I'm the one who started writing that page in September and I just kicked my lazy ass to at least complete the example. Have a look! J-D On Tue, Jan 13, 2009 at 6:06 AM, shiraz memon shiraz.li...@googlemail.comwrote: Hi, I am new to HBase and found it very interesting in terms of

RE: How can I convert from InputStream to Byte[]

2009-01-13 Thread Jim Kellerman (POWERSET)
Well, you could wrap InputStream with either BufferedInputStream (and use BufferedInputStream.read(byte[] b, int off, int len)) or wrap InputStream with a DataInputStream (and use either DataInputStream.read(byte[] b) or DataInputStream.read(byte[] b, int off, int len)) In either case above, if

RE: How can I convert from InputStream to Byte[]

2009-01-13 Thread Jim Kellerman (POWERSET)
Rather than managing your own byte arrays, you could also just have a stack local byte array of some size, read from BufferedInputStream or DataInputStream into your stack local array and write it into a ByteArrayOutputStream. When you are done you can use ByteArrayOutputStream.toByteArray() to

hbase cluster on EC2

2009-01-13 Thread Yair Even-Zohar
I took the old hbase 0.2.0 EC2 scripts and modified them to work with Hadoop 0.18.2 / hbase 0.18.1. Things seemed to be working and I can see the cluster with the different regions etc. of both the hadoop port 50030 and hbase 60010. I will release these scripts once they are working properly.

Re: regarding datamodel inside HBase

2009-01-13 Thread Jim Twensky
Shiraz, If you would like to read some more on what you can do with Hbase and compare it to an RDBMS you may also find this article helpful: http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable Jim On Tue, Jan 13, 2009 at 10:16 AM, Jean-Daniel Cryans

HBase Configuration Parameters

2009-01-13 Thread Larry Compton
I'm writing an HBase client application and I would like to be able to configure all of the HBase configuration parameters without having to supply the XML configuration files. I'm interfacing with HBase from within a JEE application and I need to be able to access multiple HBase servers.

Re: HBase Configuration Parameters

2009-01-13 Thread Larry Compton
It looks like I answered my own question. I did some additional investigation and it appears that the only parameter that needs to be changed is hadoop.master. That being said, are the HBaseConfiguration parameters documented anywhere? On Tue, Jan 13, 2009 at 2:40 PM, Larry Compton

Re: HBase Configuration Parameters

2009-01-13 Thread Jean-Daniel Cryans
Larry, The HBaseConfiguration parameters are currently documented in conf/hbase-default.xml of your hbase distro. Surely not the best solution. J-D On Tue, Jan 13, 2009 at 3:59 PM, Larry Compton lawrence.comp...@gmail.comwrote: It looks like I answered my own question. I did some additional

Frequent downs of region server

2009-01-13 Thread Edward J. Yoon
During write operation in reduce phase, region servers are killed. (64,000 rows with 10,000 columns, 3 node) 09/01/14 13:07:59 INFO mapred.JobClient: map 100% reduce 36% 09/01/14 13:11:38 INFO mapred.JobClient: map 100% reduce 33% 09/01/14 13:11:38 INFO mapred.JobClient: Task Id :

Re: Frequent downs of region server

2009-01-13 Thread stack
Edward J. Yoon wrote: During write operation in reduce phase, region servers are killed. (64,000 rows with 10,000 columns, 3 node) 10k columns is probably over what hbase is currently able to do (hbase-867). You've seen the notes at end of the

Re: Missing file

2009-01-13 Thread stack
What Andrew said and then, you might also look back in your logs to see if you can find where the region 743469791 first gets exceptions. Grep 743469791 in your master to find the unencoded name and use this grepping the master to find which host it was last deployed to. Then go there and grep