Designing HBase schema to best support specific queries

2012-01-24 Thread David Ross
Howdy, I have an HBase schema-design related question. The problem is fairly simple - I am storing notifications in hbase, each of which has a status (new, seen, and read). Here are the API's I need to provide: - Get all notifications for a user - Get all new notifications for a user - Get the

Re: Disk Seeks and Column families

2012-01-24 Thread Jason Frantz
On Tue, Jan 24, 2012 at 11:45 AM, Praveen Sripati praveensrip...@gmail.comwrote: Thanks for the response. I am just getting started with HBase. And before getting into the code/api level details, I am trying to understand the problem area HBase is trying to address through it's

Re: ANN: HBase 0.92.0 is available for download

2012-01-24 Thread Ioan Eugen Stan
Pe 24.01.2012 01:57, Stack a scris: Your HBase crew are pleased to announce the release of HBase 0.92.0. Download it from your favorite Apache mirror [1]. HBase 0.92.0 includes a wagon-load of new features including coprocessors, security, a new (self-migrating) file format, distributed log

hadoop and hbase mode

2012-01-24 Thread sangeetha k
Hi,   I wanted to setup Hadoop and Hbase in one remote server and Hbase client running in another system.   Please let me know which mode should the Hadoop and Hbase be running in the remote server. Pseudo-distributed or distributed mode?   Previously I had Hadoop, Hbase and hbase client all

Re: hadoop and hbase mode

2012-01-24 Thread Harsh J
You need a distributed mode, with no server binding to localhost. On Tue, Jan 24, 2012 at 5:21 PM, sangeetha k get2sa...@yahoo.com wrote: Hi, I wanted to setup Hadoop and Hbase in one remote server and Hbase client running in another system. Please let me know which mode should the Hadoop

Re: hadoop and hbase mode

2012-01-24 Thread sangeetha k
Harsh,   Thanks for the response.   Do I need to setup both hadoop and Hbase in distributed mode?   Thanks, Sangeetha K From: Harsh J ha...@cloudera.com To: user@hbase.apache.org; sangeetha k get2sa...@yahoo.com Sent: Tuesday, January 24, 2012 5:31 PM Subject:

Re: hadoop and hbase mode

2012-01-24 Thread yonghu
yes On Tue, Jan 24, 2012 at 2:01 PM, sangeetha k get2sa...@yahoo.com wrote: Harsh, Thanks for the response. Do I need to setup both hadoop and Hbase in distributed mode? Thanks, Sangeetha K From: Harsh J ha...@cloudera.com To: user@hbase.apache.org;

Re: hadoop and hbase mode

2012-01-24 Thread sangeetha k
Ok. thanks. let me do the setup and come back if I face any issues.   From: yonghu yongyong...@gmail.com To: user@hbase.apache.org; sangeetha k get2sa...@yahoo.com Sent: Tuesday, January 24, 2012 6:33 PM Subject: Re: hadoop and hbase mode yes On Tue, Jan 24,

Re: hadoop and hbase mode

2012-01-24 Thread sangeetha k
Hi,   what does this 'no server binding to localhost' means. Any configuration changes for this?   Thanks, Sangeetha K From: Harsh J ha...@cloudera.com To: user@hbase.apache.org; sangeetha k get2sa...@yahoo.com Sent: Tuesday, January 24, 2012 5:31 PM Subject:

Re: early memstore flushings

2012-01-24 Thread Mikael Sitruk
Your memstore may be flushed also in case the number of HLog reach its maximum, in this case your memstore may not reach the max size allocated for it. Mikael.S On Tue, Jan 24, 2012 at 1:00 AM, Jean-Daniel Cryans jdcry...@apache.orgwrote: It flushes when it reaches the memstore size, not plus

Re: hadoop and hbase mode

2012-01-24 Thread Harsh J
Mostly just means that your ZooKeeper's zoo.cfg, Hadoop's core-site.xml/mapred-site.xml, and HBase's hbase-site.xml - must *not* carry localhost (or anything that resolves to 127.0.0.1) in it anywhere, but rather use a hostname that identifies it, and is accessible by your client node. On Tue,

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Fei Dong
Thanks Stack, On Tue, Jan 24, 2012 at 1:07 AM, Stack st...@duboce.net wrote: On Mon, Jan 23, 2012 at 5:32 PM, Fei Dong dongfei...@gmail.com wrote: Hello guys, I setup a Hadoop and HBase in EC2. My Settings as follows: Apache Official Version Hadoop 0.20.203.0 HBase won't work on this

Re: ANN: HBase 0.92.0 is available for download

2012-01-24 Thread Stack
On Tue, Jan 24, 2012 at 3:10 AM, Ioan Eugen Stan stan.ieu...@gmail.com wrote: When is it going to be on Maven Central? I am on it. In next day or so. St.Ack

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Fei Dong
When I install hadoop-0.20.205.0 , the namenode can not start. [root@ip-10-114-45-186 logs]# /usr/local/hadoop-0.20.205.0/bin/start-dfs.sh starting namenode, logging to /mnt/hadoop/logs/hadoop-root-namenode-ip-10-114-45-186.out ip-10-12-55-242.ec2.internal: starting datanode, logging to

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Harsh J
Try starting it as a regular, non-root user. This is an issue with 0.20.205 scripts, perhaps fixed in more recent releases, but you do not want to be running Hadoop as root anyway. On Tue, Jan 24, 2012 at 10:22 PM, Fei Dong dongfei...@gmail.com wrote: When I install hadoop-0.20.205.0 , the

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Stack
On Tue, Jan 24, 2012 at 8:11 AM, Fei Dong dongfei...@gmail.com wrote: It says only Hadoop 0.20.205.x can match? No. Also includes 1.0.0 and other hadoop offerings (read through that section) I did not run any application before, so it should not have concurrent problem. Then I set it in

Possible ZooKeeper Connection Leak in TableOutputFormat

2012-01-24 Thread Shawn Quinn
Hello, Our application runs Map/Reduce tasks fairly frequently against HBase (Cloudera distribution 0.90.4), and we're making using of the default org.apache.hadoop.hbase.mapreduce.TableOutputFormat class for the reduce step which the TableMapReduceUtil.initTableReducerJob() sets up. We invoke

Re: Possible ZooKeeper Connection Leak in TableOutputFormat

2012-01-24 Thread Ted Yu
Thanks for reporting this, Shawn. Do you want to try out HBASE-4508 which is in HBase 0.90.5 ? On Tue, Jan 24, 2012 at 9:15 AM, Shawn Quinn squ...@moxiegroup.com wrote: Hello, Our application runs Map/Reduce tasks fairly frequently against HBase (Cloudera distribution 0.90.4), and we're

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Fei Dong
Hi Stack On Tue, Jan 24, 2012 at 12:11 PM, Stack st...@duboce.net wrote: On Tue, Jan 24, 2012 at 8:11 AM, Fei Dong dongfei...@gmail.com wrote: It says only Hadoop 0.20.205.x can match? No.  Also includes 1.0.0 and other hadoop offerings (read through that section) I did not run any

Re: ANN: HBase 0.92.0 is available for download

2012-01-24 Thread Ioan Eugen Stan
2012/1/24 Stack st...@duboce.net: On Tue, Jan 24, 2012 at 3:10 AM, Ioan Eugen Stan stan.ieu...@gmail.com wrote: When is it going to be on Maven Central? I am on it.  In next day or so. St.Ack Thanks St.Ack, please also deploy with sources, java docs and testing classifier. Having to debug

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Fei Dong
Hi Stack, I try ./zkCli.sh -server 10.114.45.186:2181, it works. I will list hbase-site.xml, zookeeper log, and Hadoop logs. Could you take a look? Thanks. hbase-site.xml: property namehbase.zookeeper.property.clientPort/name value2181/value /property property

Re: Possible ZooKeeper Connection Leak in TableOutputFormat

2012-01-24 Thread Shawn Quinn
Thanks for the quick response Ted. It does look like HBASE-4508 would resolve the issue I'm seeing, so I appreciate the pointer. We're back up and running ok for now with a custom TableOutputFormat (and custom TableInputFormat) to work around this issue, but when we're ready to move to a later

HbQL Performance Hbase Example

2012-01-24 Thread Dalia Sobhy
Dear a, I have asked this question before, but nobody answered except for one I want to determine the performance of HbQL (SQL Library for Hbase), because I need to use it in my application... If you don't know, so kindly provide me how to translate a table of in SQL CREATE TABLE

Re: HbQL Performance Hbase Example

2012-01-24 Thread Doug Meil
Hi there- I believe the consistent response to HbQL questions on the dist-list is that the package is not maintained anymore. re: so kindly provide me how to translate a table You need to read this... http://hbase.apache.org/book.html#datamodel http://hbase.apache.org/book.html#schema On

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Stack
On Tue, Jan 24, 2012 at 12:10 PM, Fei Dong dongfei...@gmail.com wrote: I test the hbase shell on another machine, which can put, get record successfully. So I guess Zookeeper is running. Whats difference between two machines? It is weird that it does not mention any path or class name behind

HBase Export

2012-01-24 Thread Greg Pelly
Hi, I'm investigating the import/export functionality in HBase, there's a lot of hits on searches but every page I hit seems to assume that I have hadoop installed as well as HBase (including the book, the definitive guide). Is it really necessary for me to have Hadoop installed in order to use

Re: HBase Export

2012-01-24 Thread Harsh J
The import/export tools are mapreduce programs but you can also run them off Hadoop's local mode (default, when not configured), not needing Hadoop setup as a cluster. HBase ships a hadoop jar in its distribution by default, which should suffice. On Wed, Jan 25, 2012 at 8:30 AM, Greg Pelly

Re: RegionServer dying every two or three days

2012-01-24 Thread Leonardo Gamas
Thanks Matt for point out the problems that can happen, I will look into it. And thanks Neil for sharing more details about your infrastructure, it has been of great help. I will run some tests with these instances and make my choice according to our needs. Thank you all for the time dispensed. :)

Re: HBase Export

2012-01-24 Thread Greg Pelly
Thanks for that, I'm now backing up. The resulting file format isn't very helpful other than if the requirement is to restore to the same or another machine. My real investigation is not just to backup it's to back up, modify the tables ie add a column family, add some more tables etc. and then

Re: NoClassDefFoundError when running Hadoop with HBase

2012-01-24 Thread Fei Dong
On Tue, Jan 24, 2012 at 5:01 PM, Stack st...@duboce.net wrote: On Tue, Jan 24, 2012 at 12:10 PM, Fei Dong dongfei...@gmail.com wrote: I test the hbase shell on another machine, which can put, get record successfully. So I guess Zookeeper is running. Whats difference between two machines?

Re: Closed parent region present in Hlog.lastSeqWritten

2012-01-24 Thread Ted Yu
Can you provide more detail on this 0.90.5 +, please ? Did it include HBASE-5225 ? Thanks On Tue, Jan 24, 2012 at 8:55 PM, bijieshan bijies...@huawei.com wrote: Hi all, We found so many hlogs in our cluster, after some analysis, we also found one splitted region occurred in

Re: Closed parent region present in Hlog.lastSeqWritten

2012-01-24 Thread bijieshan
Not including HBASE-5225(before 0.90.6.rc0). I think no relationship with this. Thanks. Jieshan -邮件原件- 发件人: Ted Yu [mailto:yuzhih...@gmail.com] 发送时间: 2012年1月25日 13:18 收件人: d...@hbase.apache.org 抄送: user@hbase.apache.org; Chenjian 主题: Re: Closed parent region present in

RE: Closed parent region present in Hlog.lastSeqWritten

2012-01-24 Thread Ramkrishna.S.Vasudevan
HBASE-5225 was merged for different purpose. But this issue does not seem to be because of HBASE-5225. Because in HBASE-5225 we tried to fix the case where the seq id is missed. Here it is not missed it is still there in lastSeqWritten without getting flushed. Regards Ram -Original

Re: Closed parent region present in Hlog.lastSeqWritten

2012-01-24 Thread bijieshan
Ted: Below is the search info for 20312224 20312223 20312225. We also analyzed this but got nothing:( [20312224] 00:31:42,529 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded

Create hbase table using script

2012-01-24 Thread Stuti Awasthi
Hi all, I am trying to create hbase tables using shell script. The table name will come as argument to the script like: $HBASE_HOME/bin/hbase shell created.sh test1 Here test1 is the table name. When in my script I want to access as normal argument I get the error: Script: Name=$1 create