This is the table that stores information about all the tables. It is
normal when a cluster is recovering for reads to be high on this table
while all the table information is being loaded into the regionservers.
http://hbase.apache.org/book/arch.catalog.html
-Pere
On Mon, Dec 15, 2014 at 12:21
Hi there,
Recently I have been experiencing instability when scanning our HBASE cluster.
The table we are trying to scan is 1.5B records 1TB, we have 12GB heap and 17
servers. Our GC options are as so:
-XX:OnOutOfMemoryError=kill -9 %p -Xmx12000m -XX:+UseConcMarkSweepGC -Xmx12g
-Xmx12g
The
I think it may be a thrift issue, have you tried playing with the connection
queues?
set hbase.thrift.maxQueuedRequests to 0
From Varun Sharma:
If you are opening persistent connections (connections that never close), you
should probably set the queue size to 0. Because those connections will
regions.
Cheers
On Wed, Nov 5, 2014 at 11:20 PM, Pere Kyle p...@whisper.sh wrote:
Watching closely a region server in action. It seems that the memstores
are being flushed at around 2MB on the regions. This would seem to
indicate that there is not enough heap for the memstore and I am hitting
all the logs.
Cheers
On Thu, Nov 6, 2014 at 10:05 AM, Pere Kyle p...@whisper.sh wrote:
So I have set the heap to 12Gb and the memstore limit to upperLimit .5
lowerLimit .45. I am not seeing any changes in behavior from the cluster so
far, i have restarted 4/17 region servers. Do I need
hours in my cluster, and
may tune it up further. It made a big impact on the number of minor
compactions running throughout the day.
On Thu, Nov 6, 2014 at 2:14 PM, Pere Kyle p...@whisper.sh wrote:
Thanks again for your help!
I do not see a single entry in my logs for memstore pressure
different setups, even the c3 level would be great if you can shrink your
disk size at all (compression and data block encodings).
On Thu, Nov 6, 2014 at 2:31 PM, Pere Kyle p...@whisper.sh wrote:
So set this property?
property
namehbase.regionserver.optionalcacheflushinterval/name
value4320
Bryan,
Thanks again for the incredibly useful reply.
I have confirmed that the callQueueLen is in fact 0, with a max value of 2 in
the last week (in ganglia)
hbase.hstore.compaction.max was set to 15 on the nodes, from a previous 7.
Freezes (laggy responses) on the cluster are frequent and
directly or as an example for adding tracing to
your application.
On Thursday, November 6, 2014, Pere Kyle p...@whisper.sh wrote:
Bryan,
Thanks again for the incredibly useful reply.
I have confirmed that the callQueueLen is in fact 0, with a max value of 2
in the last week (in ganglia
it seems yet again that it
is a bad idea to have one drive per machine, I will eventually migrate these
instances to I2
Regards,
Pere
On Nov 6, 2014, at 4:20 PM, Pere Kyle p...@whisper.sh wrote:
So I have another symptom which is quite odd. When trying to take a snapshot
of the the table
Hello,
Recently our cluster which has been running fine for 2 weeks split to 1024
regions at 1GB per region, after this split the cluster is unusable. Using the
performance benchmark I was getting a little better than 100 w/s, whereas
before it was 5000 w/s. There are 15 nodes of m2.2xlarge
(such as HBase release) ?
If you pastebin one of the region servers' log, that would help us
determine the cause.
Cheers
On Wed, Nov 5, 2014 at 9:29 PM, Pere Kyle p...@whisper.sh wrote:
Hello,
Recently our cluster which has been running fine for 2 weeks split to 1024
regions at 1GB per
...@gmail.com wrote:
Can you provide a bit more information (such as HBase release) ?
If you pastebin one of the region servers' log, that would help us
determine the cause.
Cheers
On Wed, Nov 5, 2014 at 9:29 PM, Pere Kyle p...@whisper.sh wrote:
Hello,
Recently our cluster which has
= 15, SPLITALGO =
'HexStringSplit'}
In 0.94.18, there isn't online merge. So you have to use other method to
merge the small regions.
Cheers
On Wed, Nov 5, 2014 at 10:14 PM, Pere Kyle p...@whisper.sh wrote:
Ted,
Thanks so much for that information. I now see why this split too
Nishanth,
In my experience the only way I have been able to clear the dead region
servers is to restart the master daemon.
-Pere
On Mon, Nov 3, 2014 at 9:49 AM, Nishanth S nishanth.2...@gmail.com wrote:
Hey folks,
How do I remove a dead region server?.I manually failed over the hbase
Hi,
I am implementing disaster recovery for our Hbase cluster and had one quick
question about import/export of the s3n file system.
I know that ExportTable can be given a start time and end time enabling
incremental backups. My question is how to properly store these incremental
backups on
16 matches
Mail list logo
