[
https://issues.apache.org/jira/browse/PHOENIX-34?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897586#comment-13897586
]
Lars Hofhansl commented on PHOENIX-34:
--------------------------------------
I think giving Phoenix much more than 10% by default will likely break a busy
HBase when Phoenix begins to use it. 50% is more than we give the memstores or
the blockcache! The two most important structures that HBase manages.
I am pretty certain that it will not work on a production system that is in use
(i.e. memstores and block cache mostly filled and multiple requests going on at
the same time). It'll lead to surprises and make people unhappy :)
With a reasonable heap of 10G or more it'll be a lot of space for Phoenix. Like
with HBase scale should come with many machines.
Why not make 10% the default and allow increasing? That way somebody in the
know can increase this and at the same time adjust HBase settings.
Yeah, 10k rows might is pretty small. Maybe there should not be a limit on the
number of rows anyway, but only on size (and you probably have that already),
since that is the amount that needs to be shipped to every involved
RegionServer. Presumably you would not want that to exceed a few megabytes.
I'll file jira for the other idea(s) I have.
> Insufficient memory exception on join when RHS rows count > 250K
> -----------------------------------------------------------------
>
> Key: PHOENIX-34
> URL: https://issues.apache.org/jira/browse/PHOENIX-34
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 3.0.0
> Environment: HBase 0.94.14, r1543222, Hadoop 1.0.4, r1393290, 2 RS +
> 1 Master, Heap 4GB per RS
> Reporter: Mujtaba Chohan
> Fix For: 3.0.0
>
>
> Join fails when rows count of RHS table is >250K. Detail on table schema is
> and performance numbers with different LHS/RHS row count is on
> http://phoenix-bin.github.io/client/performance/phoenix-20140210023154.htm.
> James comment:
> So that's with a 4GB heap allowing Phoenix to use 50% of it. With a pretty
> narrow table: 3 KV columns of 30bytes. Topping out at 250K is a bit low. I
> wonder if our memory estimation matches reality.
> What do you think Maryann?
> How about filing a JIRA, Mujtaba. This is a good conversation to have on the
> dev list. Can we move it there, please?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
