hbase RegionServer dead suddenly

2011-12-30 Thread xiaochao
Hello: hbase RegionServer is dead with the log below.can someone help me? 2011-12-28 12:12:34,899 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server serverName=192.168.3.39,50012,1324979072340, load= (requests=403, regions=1398, usedHeap=3731, maxHeap=9983): Replay

Re: No. of families

2011-12-30 Thread Michel Segel
Sorry, but you misunderstand. Implementing a hierarchical model in any NoSQL database is trivial. Sent from a remote device. Please excuse any typos... Mike Segel On Dec 30, 2011, at 9:01 PM, Imran M Yousuf wrote: > Hi Michael, > > I totally agree. Thats what we tried to implement in Sma

Re: No. of families

2011-12-30 Thread Imran M Yousuf
Hi Michael, I totally agree. Thats what we tried to implement in Smart CMS making it easy for clients to retrieve and persist data with more added facility. BTW, I forgot to mention that Smart CMS persists data in HBase. Regards, Imran On Sat, Dec 31, 2011 at 8:56 AM, Michael Segel wrote: > Hi

Re: No. of families

2011-12-30 Thread Michael Segel
Hierarchical data doesn't necessarily has anything to do w column families. You can do a hierarchical model in a single column family. It's pretty straight forward. Sent from my iPhone On Dec 30, 2011, at 6:34 PM, "Imran M Yousuf" wrote: > Hi, > > Rather than addressing the issue of how many

Re: No. of families

2011-12-30 Thread Imran M Yousuf
Hi, Rather than addressing the issue of how many column family may be used and query performance on them, I would like to address the problem of hierarchical data. We were facing an issue of storing hierarchical data in one of our applications and for solving that, and many other features, we tur

Re: ANN: HBase 0.90.5 available for download

2011-12-30 Thread Ted Yu
When integrating HBASE-5102 I found the following issue which would leave stale connection hanging around if you use HBaseAdmin. This only happens when "hbase.connection.per.config" carries value of false. In 0.90.5, its default value is true. We're discussing releasing 0.90.6 in January. Stay tu

Re: Do I really need fully-distributed mode for HBase?

2011-12-30 Thread Stack
On Fri, Dec 30, 2011 at 2:17 PM, Mark Kerzner wrote: > I found the answer, here it is for everyone's benefit, > Thanks for updating the list Mark, St.Ack

Re: Do I really need fully-distributed mode for HBase?

2011-12-30 Thread Mark Kerzner
I found the answer, here it is for everyone's benefit, If you want to access HBase from a remote client (like any hadoop node, for example), then you run into this problem: HBase master auto-discovers itself as localhost (using ‘hostname’ command, and tells this to zookeeper, which tells this to y

Do I really need fully-distributed mode for HBase?

2011-12-30 Thread Mark Kerzner
Hi, if I am running HBase in pseudo-distributed mode and I cannot connect to it from other nodes, I guess, that is to be expected, right? I mean, the browser connects to http://hbase-url:60010, but the zookeeper, which ignored my ip and started on localhost, would give me a problem. So I would ne

Re: Could an EC2 machine to 4 times slower than local dev workstation?

2011-12-30 Thread Li Pi
Yup. Virtualized IO pretty much explains it. On Thu, Dec 29, 2011 at 8:20 PM, Mark Kerzner wrote: > Hi, > > I am running a small program to load about 1 million rows into HBase. It > takes 200 seconds on my dev machine, and 800 seconds on a c1.medium EC2 > machine. Both are running the same versi

Re: HBase support in SHDP Was: Hadoop with Spring / Guice

2011-12-30 Thread Costin Leau
On 12/30/2011 12:43 PM, Ted Yu wrote: > Costin: > Please include user@hbase.apache.org in > your reply. > My discussion would mostly be HBase-related. # included hbase though I'm not subscribed to that list so I don't think my messages will go through. > > I guess

Re: Could an EC2 machine to 4 times slower than local dev workstation?

2011-12-30 Thread Michel Segel
Hi, Yes the performance hit is normal. Looks like you're seeing network latency on disk I/O. Could also be a tuning issue. (differences in configurations...) Not sure how much. CPU difference will impact performance, while disk I/O will really kill you. Sent from a remote device. Please excuse

Re: RAM Problems - Keeps Crashing

2011-12-30 Thread Seraph Imalia
Setting that property to false has not made any difference, hbase has just crashed again (ran out of heap) and I am busy restarting it. What do I do now? On 29 Dec 2011, at 5:56 PM, Seraph Imalia wrote: > Thanks, > > I will try disabling it to see if the memory is being taken up by MSLAB. >

Re: Could an EC2 machine to 4 times slower than local dev workstation?

2011-12-30 Thread Mark Kerzner
Thank you, Bryan, that is very important and clear some cloudiness in my mind. Sincerely, Mark On Fri, Dec 30, 2011 at 10:54 AM, Bryan Beaudreault < bbeaudrea...@hubspot.com> wrote: > We have also seen this in our testing, though we focused mainly on MR more > than HBase. > > Keep in mind that

Re: No. of families

2011-12-30 Thread Doug Meil
The origin on the lower number of CF's wasn't so much on the reads but on the writes (and GC churn) that happens with MemStore flushes per Store. On 12/29/11 1:18 AM, "lars hofhansl" wrote: >Less is not necessarily better. HBase can ignore stores (column families) >during a scan or get if th

Re: Could an EC2 machine to 4 times slower than local dev workstation?

2011-12-30 Thread Bryan Beaudreault
We have also seen this in our testing, though we focused mainly on MR more than HBase. Keep in mind that EC2 Compute Units are defined as follows: The amount of CPU that is allocated to a particular instance is expressed > in terms of these EC2 Compute Units. We use several benchmarks and tests t

Re: No. of families

2011-12-30 Thread Lars H
I think the code in ScanQueryMatcher would suggest a performance limit somewhere in the high 10k's of columns. This is a completely unscientific guess just from looking at the code. Jesse Yates schrieb: >Out of curiosity (havent rtfm on this yet) do we have any hard >bounds/performance impact

Re: Connecting to google region server

2011-12-30 Thread Santiago Vázquez
Harsh : No, these hostnames are not mine. I don't think that this is a DNS issue because i have only local addresses in the configuration files. Stask: I'm getting this messages only when hbase is down. I'm newbie at hbase and i am still fighting with some problems at startup. Sometimes the cluste

Re: No. of families

2011-12-30 Thread Jesse Yates
Out of curiosity (havent rtfm on this yet) do we have any hard bounds/performance impact on the max number of column families/qualifiers? Has that behavior changed with the dynamic cf stuff that fairly recently got rolled in? Further, any pointers on where to start digging into the code on thi

Re: NPE while obtaining row lock

2011-12-30 Thread Yves Langisch
Still happens but before I'm going to add some debugging information I'll try to deploy the new version 0.90.5. - Yves On Dec 18, 2011, at 12:08 AM, Stack wrote: > On Fri, Dec 16, 2011 at 8:20 AM, Yves Langisch wrote: >> I'm using the async hbase client (1.0) and there is no way to choose a >

Re: HBase support in SHDP Was: Hadoop with Spring / Guice

2011-12-30 Thread Ted Yu
Costin: Please include user@hbase.apache.org in your reply. My discussion would mostly be HBase-related. I guess you're dealing with HBase 0.90.4 where connection sharing provided by HBASE-3777 is absent. HBase 0.90.5, just released, contains HBASE-4508 which backports HBASE-3777 to 0.90 codebase.

HBase support in SHDP Was: Hadoop with Spring / Guice

2011-12-30 Thread Ted Yu
Hi, Costin: I work on HBase. I went over http://static.springsource.org/spring-hadoop/docs/current/reference/hbase.htmlbut didn't have time to download the source code. Is there a typo: 'does more then easily' Should 'then' be 'than' ? For the following config: May I ask what would the proxies

HBase Custom Comparator

2011-12-30 Thread Nageswaran Surendhar
public class CustomComparator extends WritableByteArrayComparable { final CimKeyType cimKeyType; final byte[] value; public CustomComparator(byte[] value, CimKeyType cimKeyType) { super(value); this.value = value; this.cimKeyType = cimKeyType;