We haven't disable block cache. So I doubt that's the problem.
On Mon, Jul 8, 2013 at 4:50 PM, Varun Sharma wrote:
> FYI, if u disable your block cache - you will ask for "Index" blocks for
> every single request. So such a high rate of request is plausible for Index
> blocks even when your requ
meta blocks are at the end:
http://hbase.apache.org/book.html#d2617e12979, a way to tell would be
by logging from the HBase side but then I guess it's hard to reconcile
with which file we're actually reading from...
Regarding your second question, you are asking if we block HDFS
blocks? We don't,
FYI, if u disable your block cache - you will ask for "Index" blocks for
every single request. So such a high rate of request is plausible for Index
blocks even when your requests are totally random on your data.
Varun
On Mon, Jul 8, 2013 at 4:45 PM, Viral Bajaria wrote:
> Good question. When I
Good question. When I looked at the logs, it's not clear from it whether
it's reading a meta or data block. Is there any kind of log line that
indicates that ? Given that it's saying that it's ready from a startOffset
I would assume this is a data block.
A question that comes to mind, is this read
Do you know if it's a data or meta block?
J-D
On Mon, Jul 8, 2013 at 4:28 PM, Viral Bajaria wrote:
> I was able to reproduce the same regionserver asking for the same local
> block over 300 times within the same 2 minute window by running one of my
> heavy workloads.
>
> Let me try and gather so
I was able to reproduce the same regionserver asking for the same local
block over 300 times within the same 2 minute window by running one of my
heavy workloads.
Let me try and gather some stack dumps. I agree that jstack crashing the
jvm is concerning but there is nothing in the errors to know w
On Mon, Jul 8, 2013 at 12:22 PM, Viral Bajaria wrote:
> - I tried taking a stack trace using jstack but after the dump it crashed
> the regionserver. I also did not take the dump on the offending
> regionserver, rather took it on the regionservers that were making the
> block count. I will take a
Thanks guys for going through that never-ending email! I will create the
JIRA for block cache eviction and the regionserver assignment command. Ted
already pointed to the JIRA which tries to go a different datanode if the
primary is busy (I will add comments to that one).
To answer Andrews' questi
ect: optimizing block cache requests + eviction
Hi,
TL;DR;
Trying to make a case for making the block eviction strategy smart and to
not evict remote blocks more frequently and make the requests more smarter.
The question here comes after I debugged the issue that I was having with
random region serv
> Would it make sense to give remote blocks higher priority over the local
blocks that can be read via SCR and not let them get evicted if there is a
tie in which block to evict ?
That sounds like a reasonable idea. As are the others.
But first, could this be a bug?
What version of HBase? Were
For suggestion #3 below, take a look at:
HBASE-7509 Enable RS to query a secondary datanode in parallel, if the
primary takes too long
Cheers
On Mon, Jul 8, 2013 at 3:04 AM, Viral Bajaria wrote:
> Hi,
>
> TL;DR;
> Trying to make a case for making the block eviction strategy smart and to
> not e
Hi,
TL;DR;
Trying to make a case for making the block eviction strategy smart and to
not evict remote blocks more frequently and make the requests more smarter.
The question here comes after I debugged the issue that I was having with
random region servers hitting high load averages. I initially
12 matches
Mail list logo
