Original Message-
>> From: Anoop Sam John [mailto:anoo...@huawei.com]
>> Sent: Wednesday, August 22, 2012 5:35 AM
>> To: user@hbase.apache.org; J Mohamed Zahoor
>> Subject: RE: Using HBase serving to replace memcached
>>
>>> I could be wrong. I think HFi
; then scan each one of them from start to end?
>
> Thanks,
> Abhishek
>
>
> -Original Message-
> From: Anoop Sam John [mailto:anoo...@huawei.com]
> Sent: Wednesday, August 22, 2012 5:35 AM
> To: user@hbase.apache.org; J Mohamed Zahoor
> Subject: RE: Using HBa
To: user@hbase.apache.org; J Mohamed Zahoor
Subject: RE: Using HBase serving to replace memcached
Great explanation. May be diverging from the thread's original question, but
could you also care to explain the difference if any, in searching for a
rowkey [ that you mentioned below ] Vs searching
hamed Zahoor
Subject: RE: Using HBase serving to replace memcached
> I could be wrong. I think HFile index block (which is located at the
> end
>> of HFile) is a binary search tree containing all row-key values (of
>> the
>> HFile) in the binary search tree. Search
.
>
> -Anoop-
> ________________
> From: Lin Ma [lin...@gmail.com]
> Sent: Wednesday, August 22, 2012 5:41 PM
> To: J Mohamed Zahoor; user@hbase.apache.org
> Subject: Re: Using HBase serving to replace memcached
>
> Thanks Zahoor,
>
> I read thro
On Wed, Aug 22, 2012 at 6:28 AM, Lin Ma wrote:
> Thanks Anoop,
>
> My question is answered. Are you writing related part of code in HBase?
> From your detailed and knowledgeable description, you seems to be the
> author. :-)
>
Anoop did not write that particular piece of code. He has though
mad
e such that 'x' comes in between and Hbase
> will load that block. So usage of blooms can avoid this IO. Hope this is
> clear for you now.
>
> -Anoop-
> ________________
> From: Lin Ma [lin...@gmail.com]
> Sent: Wednesday, August 22, 2012 5:41 PM
> To: J Mohamed Zahoor;
ooms can avoid this IO. Hope this is clear for you now.
-Anoop-
From: Lin Ma [lin...@gmail.com]
Sent: Wednesday, August 22, 2012 5:41 PM
To: J Mohamed Zahoor; user@hbase.apache.org
Subject: Re: Using HBase serving to replace memcached
Thanks Zahoor,
I read through the document y
Thanks Zahoor,
I read through the document you referred to, I am confused about what means
leaf-level index, intermediate-level index and root-level index. It is
appreciate if you could give more details what they are, or point me to the
related documents.
BTW: the document you pointed me is very
>
> I could be wrong. I think HFile index block (which is located at the end
> of HFile) is a binary search tree containing all row-key values (of the
> HFile) in the binary search tree. Searching a specific row-key in the
> binary search tree could easily find whether a row-key exists (some node i
Thanks Zahoor,
> If there is no bloom... you have to load every block and scan to find if
the row exists..
I could be wrong. I think HFile index block (which is located at the end of
HFile) is a binary search tree containing all row-key values (of the HFile)
in the binary search tree. Searching a
>
>
> 1. After reading the materials you sent to me, I am confused how Bloom Filter
> could save I/O during random read. Supposing I am not using Bloom Filter, in
> order to find whether a row (or row-key) exists, we need to scan the index
> block which is at the end part of an HFile, the scan
Thank you Zahoor,
Two more comments,
1. After reading the materials you sent to me, I am confused how Bloom
Filter could save I/O during random read. Supposing I am not using Bloom
Filter, in order to find whether a row (or row-key) exists, we need to scan
the index block which is at the end part
>
>
>
> 1. I know very basics of Bloom filters, which is used for detect whether an
> item is in a set. How to use Bloom filters in HBase to improve random read
> performance? Could you show me an example? Thanks.
This will help omit loading the blocks (thereby saving IO and cache churn)
whi
Thanks for the reply, Zahoor.
Some more comments,
1. I know very basics of Bloom filters, which is used for detect whether an
item is in a set. How to use Bloom filters in HBase to improve random read
performance? Could you show me an example? Thanks.
2. "Also more client connections is one more
Thanks Asif,
For your comments, "Which means there are some requests which will be
handled incorrectly.", could you show me an example about what do you mean
"handled incorrectly"?
regards,
Lin
On Tue, Aug 21, 2012 at 3:26 AM, Asif Ali wrote:
> I've used memcached heavily in such scenarios and
Again. if your data is so huge that it is much larger than the available
RAM, you might want to rethink.
There are some configs in HBase that will help you in random read
scenarios... like Bloom filters etc.
Also more client connections is one more issue that might infest you...
where connection po
I've used memcached heavily in such scenarios and all such data is always
in Memory.
Memcached definitely is a great solution for this and scales very well. But
keep in mind - it is not consistent. Which means there are some requests
which will be handled incorrectly.
Memcached is great but also
Thank you Drew. I like your reply, especially blocking cache nature
provided by HBase. A quick question, for traditional memcached, all of the
items are in memory, no disk is used, correct?
regards,
Lin
On Mon, Aug 20, 2012 at 9:26 PM, Drew Dahlke wrote:
> I'd say if the memcached model is work
I'd say if the memcached model is working for you, stick with it.
HBase (currently) caches whole blocks. With cache blocks enabled you
can achieve 10s of thousands of reqs/sec with a pretty small cluster.
However there's a catch. Once you reach the point where your tables
are so large they can't al
20 matches
Mail list logo
