Hi all,
we are using HBase 0.20.6 on a cluster of about 25 nodes with about 30k
regions and are experiencing as issue which causes running M/R jobs to
fail.
When we restart single RegionServer, then happens the following:
1) all regions of that RS get reassigned to remaing (say 24) nodes
2)
Hi Jan,
That area of HBase was reworked a lot in the upcoming 0.90.0 and
region opening and closing can now be done in parallel for multiple
regions.
Also, the balancer works differently and may not even assign a single
region to a new region server (or a dead one that was restarted) until
the ba
Hi Daniel,
I thought that version 0.90.0 would have major rewrites in this area,
could you give a rough estimate when the new version will be out?
Thanks,
Jan
On 13.12.2010 20:43, Jean-Daniel Cryans wrote:
Hi Jan,
That area of HBase was reworked a lot in the upcoming 0.90.0 and
region open
Hi Jan,
Any day now!
Really, there just a few little road bumps but nothing major ad once
they are resolved it will be released. Just rushing it for the sake of
releasing it will not make anyone happy (if we find issues right away
just afterwards). Please bear with us!
Lars
On Tue, Dec 14, 2010
Hi Lars,
sure, I understand this. :-)
Thanks.
On 14.12.2010 16:17, Lars George wrote:
Hi Jan,
Any day now!
Really, there just a few little road bumps but nothing major ad once
they are resolved it will be released. Just rushing it for the sake of
releasing it will not make anyone happy (if w
Hi Jan,
Thank you. I hope this did not come through derogative I really meant
this in a friendly way (emails sometimes - errr often - do not convey
this right).
Lars
On Tue, Dec 14, 2010 at 5:00 PM, Jan Lukavský
wrote:
> Hi Lars,
>
> sure, I understand this. :-)
>
> Thanks.
>
> On 14.12.2010 16
Can you do w/ less regions? 1k plus per server is pushing it I'd say.
Can you up your region sizes, for instance?
St.Ack
On Mon, Dec 13, 2010 at 8:36 AM, Jan Lukavský
wrote:
> Hi all,
>
> we are using HBase 0.20.6 on a cluster of about 25 nodes with about 30k
> regions and are experiencing as i
Hi Stack,
We have been running a small cluster (name node + 5 rs) on 0.20.3 for a
long time now. We are currently at 1100 regions per RS. As far as I can
tell, I have not seen any problems or changes in behavior due this.
What kind of problems can I expect with 1K+ regions per RS? What is a
We can give it a try. Currently we use 512 MiB per region, is there any
upper bound for this value which is not recommended to cross? Are there
any side-effects we may expect when we set this value to say 1 GiB? I
suppose at least a bit longer random gets?
Thanks,
Jan
On 14.12.2010 18:50, St
> We can give it a try. Currently we use 512 MiB per region, is there any
> upper bound for this value which is not recommended to cross?
Like I said in my first email, we recommend 1GB.
> Are there any
> side-effects we may expect when we set this value to say 1 GiB?
HBase may be faster overtim
On Tue, Dec 14, 2010 at 5:28 PM, Igor Ranitovic wrote:
> Hi Stack,
>
> We have been running a small cluster (name node + 5 rs) on 0.20.3 for a long
> time now. We are currently at 1100 regions per RS. As far as I can tell, I
> have not seen any problems or changes in behavior due this.
>
Thats a
Hi Stack,
OK, I will up it to 1GB. Do I need to do a full restart or rolling RS
restarts will do?
I have been happy with 0.20.3 even though it is an ancient release.
Given that we have a 0.3-man team dedicated to HBase it is hard to find
time for upgrades. It will have to wait for 0.90 or on
On Wed, Dec 15, 2010 at 11:44 AM, Igor Ranitovic wrote:
> Hi Stack,
>
> OK, I will up it to 1GB. Do I need to do a full restart or rolling RS
> restarts will do?
>
Rolling restart would do it.
> I have been happy with 0.20.3 even though it is an ancient release. Given
> that we have a 0.3-man t
If you did the change, can you share your experience/results?
On Wed, Dec 15, 2010 at 12:04 AM, Jan Lukavský wrote:
> We can give it a try. Currently we use 512 MiB per region, is there any
> upper bound for this value which is not recommended to cross? Are there any
> side-effects we may expect
Hi,
sure, we are experiencing the following:
- regions are getting unavailable for much less time, so clients are
no longer failing (in fact some of them usually still fail on
RetriesExhausedException caused by "failed setting up proxy", but there
are only few of them)
- on the other hand
Hi J-D,
If 1GB region size and about 1K regions per server is recommended, does
that mean that a region node should server about 1TB of compressed data
at most?
If that is the case than having more than 2TB (1TB for data and 1TB of
spare free space) is wasteful for data nodes that are part o
16 matches
Mail list logo
