Hi
I was working on way to find a substitute for coprocessor(because of
various issue with them)
so tried Custom Filters and found that the custom filter are much more
resource exhaustive (CPU) then coprocessor having single column value
filter.
*Query: Get List of Publications for UDANDA (CF:Attr
Hi Vikram,
Maybe I'm misunderstanding your question, but isn't this expected
since coprocessors do the heavy lifting at the level of the RegionServer
whereas the client API means you hit a bottleneck as data is transferred
from each RS to your gateway?
-Dima
On Monday, June 16, 2014, Vikram Sing
Hi Dima
Umm but as i know the working of CoProcessor and Custom filter are almost
same, they execute in similar fashion.
*But with Coprocessor i never got such high levels of CPU usage*. Though
Heap and JVM freeze are some issues with CoPro
On Mon, Jun 16, 2014 at 4:18 PM, Dima Spivak wrote:
>
The comparison u are making btw SCVF and a custom filter u wrote?
Or it is a CP impl vs custom filter/SCVF
Ur query seems select where cf:q1 = ?.. Wondering why u want a
custom filter here and can not use SCVF. Can u pls give some more details?
-Anoop-
On Mon, Jun 16, 2014 at 4:40 PM, Vikra
Can you format the table for better reading ?
You can put picture on third party website and post link here.
Does your custom filter maintain state ?
What hbase version are you using ?
Have you checked region server logs during the 15 minutes ?
Cheers
On Jun 16, 2014, at 3:25 AM, Vikram Singh
Hi
ya Anoop the comparison was between SCVF and Custom Filter. Seeing the
deployment process of Custom Filter, i.e building a jar deploying it on to
all RS
*i thought that the work in similar fashion as that of CP*. If they don't
process records parallel on RS why not use HDFS to distribute the jar
>> If they don't
process records parallel on RS why not use HDFS to distribute the jar?
See https://issues.apache.org/jira/browse/HBASE-1936 for dynamic class
loading of jars.
In the first mail you had mentioned that there are various problems in CP?
What are those problems? Filters are mainly us
Hi Ramkrishna
issues with CoPro there are many
1: Garbage Collection
2: JVM Freeze
3: Removal of CoPro from RS unless you do a rolling restart
4: Class unloading
5:Class Collision Issues
6:Shared JVM between CoPro and RS
there may be some more that i am not aware of :-)
On Tue, Jun 17, 2014 at
Vikram:
Discussion on coprocessor may get very long.
Let's focus on making your custom filter work.
Do you have unit test for the custom filter ?
If you limit the scan (with this filter) to one region, you should be able
to obtain jstack which would give us more clue.
Cheers
On Tue, Jun 17, 20
Hi Ted
There is no problem in execution of Custom filters the issue is why are
they more resource exhaustive.
So i have two questions
1. Are custom filter work in similar fashion as CoPro works? i.e working
parallel on all RS.
If yes why are they more resource exhaustive
2. What exactly we mean w
A normal scan is not executed parallely on regions. It is serial way. And
a Filter is a way to filter out some data during the scan. So Filter
works along with scan. There is nothing like Filter execute parellel or so.
What was working paralle is the Endpoints. So if u impl one and call a
met
Hi Anoop here is the code for Custom Filter implementation
public class MyCustomFilter extends FilterBase {
/**
* @param args
* @author vikram singh chandel
*/
private byte[] value = null;
private boolean filterRow = true;
public MyCustomFilter() {
super();
}
public MyCustomFilter(byte[] val
Not sure why u want a custom Filter for this.. This is like SCVF. Infact
simpler than SCVF.. Seems u have only one cf:column . Is it?
wrt performance
if (Bytes.compareTo(value, kv.getValue()) == 0) {
Try replacing this with
if (Bytes.compareTo(value, 0, value.length, kv.getValueArray(),
kv.ge
13 matches
Mail list logo
