IIUC, this seems to be a problem with Direct memory cleanup.
I've seen this problem mostly with application using lot of NIO
(RegionServer, DataNode).
Symptoms of this problem
1) Allocated heap is 1 GB
2) VIRT/RES will occupy 10GB or even more
3) GC logs says, it takes more than 1 minute to clean
Gen,
HBase has HA across the entire stack. Have you read the original Google
Bigtable paper to understand the architecture of the system? That is a great
place to start.
-Amandeep
On Tuesday, July 10, 2012 at 9:40 PM, Gen Liu wrote:
> Hi, I'm new here. I'm doing evaluation on Hbase before
Hi, I'm new here. I'm doing evaluation on Hbase before applying it to
production with big traffic and data.
I was search for detailed documents that specifying the auto failover behavior
of Hbase, e.g. What happen if Master/RegionServer/Zookeeper die(maybe at the
same time), how does administrat
Search for "hadoop-dns-checker" in http://hbase.apache.org/book.html
That tool might help figure out if your cluster networking is all right.
--S
On Mon, Jul 9, 2012 at 3:03 PM, Dhaval Shah wrote:
> There is definitely a debug flag on hbase.. You can find out details on
> http://hbase.apache.org
Nice discussion here.
Jean-Marc:
Do you mind logging a JIRA ?
Thanks
On Tue, Jul 10, 2012 at 3:15 PM, Michael Segel wrote:
> Nobody is arguing that the exception message doesn't make sense. +1 to
> making better error messages.
>
> What I am suggesting is to take advantage of HBase's meta data.
Nobody is arguing that the exception message doesn't make sense. +1 to making
better error messages.
What I am suggesting is to take advantage of HBase's meta data. That is, at run
time, you can query HBase to determine what tables exist, and see what column
families exist.
So you can avoid th
Hi,
When a region is loaded with a coprocessor (by loading the coprocessor to a
table), is an instance opened on each region immediately and then waits to
be executed? I ask because I am thinking about using some static variables
with my postPut RegionObservers and I am not sure if I need to have
I expect (without double checking the path in the code ;-) that the
code in HConnectionManager will retry.
On Tue, Jul 10, 2012 at 7:22 PM, Suraj Varma wrote:
> Yes.
>
> On the maxRetries, though ... I saw the code
> (http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hbase/hbase/0.90.2/o
Yes.
On the maxRetries, though ... I saw the code
(http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hbase/hbase/0.90.2/org/apache/hadoop/hbase/ipc/HBaseClient.java#677)
show
this.maxRetries = conf.getInt("hbase.ipc.client.connect.max.retries", 0);
So - looks like by default, the maxRetr
Thanks for the jira.
The client can be connected to multiple RS, depending on the rows is
working on. So yes it's initial, but it's a dynamic initial :-).
This said there is a retry on error...
On Tue, Jul 10, 2012 at 6:46 PM, Suraj Varma wrote:
> I will create a JIRA ticket ...
>
> The only side
Created https://issues.apache.org/jira/browse/HBASE-6364 for this issue.
Thanks,
--Suraj
On Tue, Jul 10, 2012 at 9:46 AM, Suraj Varma wrote:
> I will create a JIRA ticket ...
>
> The only side-effect I could think of is ... if a RS is having a GC of
> a few seconds, any _new_ client trying to co
Hello Asaf,
On Tue, Jul 10, 2012 at 02:20:03PM +0300, Asaf Mesika wrote:
> On Jul 10, 2012, at 8:57 AM, Arvid Warnecke wrote:
> > On Mon, Jul 09, 2012 at 09:10:12PM +0300, Asaf Mesika wrote:
> >> On Jul 9, 2012, at 21:00 PM, Harsh J wrote:
> >>> The hbase-daemon.sh does not ssh back into the host,
I will create a JIRA ticket ...
The only side-effect I could think of is ... if a RS is having a GC of
a few seconds, any _new_ client trying to connect would get connect
failures. So ... the _initial_ connection to the RS is what would
suffer from a super-low setting of the ipc.socket.timeout. Th
I doubt on your system load, because normally GC will not take more time to
collect from 473MB (used heap), here allocated heap just gone upto 1.2GB.
Can u check system load factor from the top command & % system wait.
What is your system configuration?
Thanks & Regards,
Gopinathan A
I tried to change the flag but yesterday it happened again:
Application time: 0.3025790 seconds
30013.866: [GC 30013.866: [ParNew: 106069K->989K(118016K), 178.8437590 secs]
473853K->369013K(1275392K), 178.8438570 secs] [Times: user=0.05 sys=178.82,
real=178.81 secs]
Total time for which applicat
+1 a proper error message always helps IMHO
--
On Tue 10 Jul, 2012 5:58 PM IST Jean-Marc Spaggiari wrote:
>Hi Michael,
>
>I agree that in the code we have access to all the information to
>access the right column.
>
>However, let's imagine the column family name is
Hi Michael,
I agree that in the code we have access to all the information to
access the right column.
However, let's imagine the column family name is dynamically retrieved
from a property file, and there is a typo. Or, another process removed
the column family. Or there is a bug in the code, an
On Mon, Jul 9, 2012 at 1:05 PM, Alex Baranau wrote:
> Hey, this is closer!
>
> However, I think I'd want to avoid major compaction. In fact I was thinking
> about avoiding any compactions & splitting.
> ...
So, you are saying that major compaction will look at max/min ts metainfo
> of the HFile a
Regardless,
Its still a bad design.
On Jul 9, 2012, at 10:02 PM, Jonathan Hsieh wrote:
> Keith,
>
> The HBASE-3584 feature is a 0.94 and we are strongly considering an 0.94
> version for for a future CDH4 update. There is very little chance this
> will get into a CDH3 release.
>
> Jon.
>
> O
On Jul 10, 2012, at 8:57 AM, Arvid Warnecke wrote:
> Hello,
>
> On Mon, Jul 09, 2012 at 09:10:12PM +0300, Asaf Mesika wrote:
>> On Jul 9, 2012, at 21:00 PM, Harsh J wrote:
>>> The hbase-daemon.sh does not ssh back into the host, so preserves any
>>> environment variables you haven't otherwise se
A network issue ?? it's weird, cause reads/writes are working well and not
rising errors (I'll double check it)
Regards
Cyril SCETBON
On Jul 9, 2012, at 10:55 PM, Jean-Daniel Cryans wrote:
> We've been running with distributed splitting here for >6 months and
> never had this issue. Also the ex
Can you paste the logs here??
Regards,
Mohammad Tariq
On Tue, Jul 10, 2012 at 2:42 PM, wrote:
> Hello Mohammad,
>
> Thanks :-) , I dont know what I am doing wrong that my HRegionServer is not
> running, I am running single node cluster,
>
> hadoop/core-site.xml
>
>
>
> fs.d
Hello Mohammad,
Thanks :-) , I dont know what I am doing wrong that my HRegionServer is not
running, I am running single node cluster,
hadoop/core-site.xml
fs.default.name
hdfs://localhost:9000
and here is hbase-site.xml
hbase.rootdir
Hello Yogesh,
There are 2 types of modes that can be configured on a single
machine viz. standalone and pseudo-distributed..RegionServer is
required in both the cases (not only in fully distributed mode). It is
the place where our data is stored.
Regards,
Mohammad Tariq
On Tue, Jul
Hello Mohammad :-)
Thanks for replying :-)
here is the
hadoop/core-site.xml
fs.default.name
hdfs://localhost:9000
and here is hbase-site.xml
hbase.rootdir
HADOOP/HBASE-0.92.1/hbase-0.92.1-security
Hbase is working but I am
Thanks Harsh :-),
I followed that url and made changes in hbase-site.xml.
And as mentioned in this page hbase is working,
I am little bit in doubt that does HRegionServer run only on distributed nodes
?
single node setup doesn't require HRegionServer if so then why?
Please suggest
Regards
Yo
Hi Asaf,
Apologies for being so dumb. I should have read the question properly.
Regards,
Mohammad Tariq
On Tue, Jul 10, 2012 at 9:01 AM, Asaf Mesika wrote:
> The int, short, short part goes to the time stamp.
>
> Thanks!
>
> Sent from my iPad
>
> On 10 ביול 2012, at 01:08, Mohammad Ta
Hi Yogesh,
Can you paste the content of your core-site.xml and
hbase-site.xml files??
Regards,
Mohammad Tariq
On Tue, Jul 10, 2012 at 1:05 PM, Harsh J wrote:
> Yogesh,
>
> Please follow the section for Pseudo-distributed and distributed
> deployment, documented here:
> http://hbase.
Yogesh,
Please follow the section for Pseudo-distributed and distributed
deployment, documented here:
http://hbase.apache.org/book.html#standalone_dist.
If you do not follow the instructions there, and lack proper
hbase-site.xml configs, your HBase starts in a standalone (Local FS)
mode and does
Hi All,
I am new to hadoop and hbase, and running Hadoop over single node cluster so as
Hbase also,
I have used command
start-hbase.sh
it only starts Hmaster
no other threads like Hregionserver
here are the changes that I have made in hbase/conf/hbase-env.sh
30 matches
Mail list logo
