Re: Scan problem
Thanks all of you, and your answer help me a lot. 2018-03-19 22:31 GMT+08:00 Saad Mufti <saad.mu...@gmail.com>: > Another option if you have enough disk space/off heap memory space is to > enable bucket cache to cache even more of your data, and set the > PREFETCH_ON_OPEN => true option on the column families you want always > cache. That way HBase will prefetch your data into the bucket cache and > your scan won't have that initial slowdown. Or if you want to do it > globally for all column families, set the configuration flag > "hbase.rs.prefetchblocksonopen" to "true". Keep in mind though that if you > do this, you should either have enough bucket cache space for all your > data, otherwise there will be a lot of useless eviction activity at HBase > startup and even later. > > Also, where a region is located will also be heavily impacted by which > region balancer you have chosen and how you have tuned it in terms of how > often to run and other parameters. A split region will stay initially at > least on the same region server but your balancer if and when run can move > it (an indeed any region) elsewhere to satisfy its criteria. > > Cheers. > > > Saad > > > On Mon, Mar 19, 2018 at 1:14 AM, ramkrishna vasudevan < > ramkrishna.s.vasude...@gmail.com> wrote: > > > Hi > > > > First regarding the scans, > > > > Generally the data resides in the store files which is in HDFS. So > probably > > the first scan that you are doing is reading from HDFS which involves > disk > > reads. Once the blocks are read, they are cached in the Block cache of > > HBase. So your further reads go through that and hence you see further > > speed up in the scans. > > > > >> And another question about region split, I want to know which > > RegionServer > > will load the new region afther splited , > > Will they be the same One with the old region? > > Yes . Generally same region server hosts it. > > > > In master the code is here, > > https://github.com/apache/hbase/blob/master/hbase- > > server/src/main/java/org/apache/hadoop/hbase/master/assignment/ > > SplitTableRegionProcedure.java > > > > You may need to understand the entire flow to know how the regions are > > opened after a split. > > > > Regards > > Ram > > > > On Sat, Mar 17, 2018 at 9:02 PM, Yang Zhang <zhang.yang...@gmail.com> > > wrote: > > > > > Hello everyone > > > > > > I try to do many Scan use RegionScanner in coprocessor, and > > ervery > > > time ,the first Scan cost about 10 times than the other, > > > I don't know why this will happen > > > > > > OneBucket Scan cost is : 8794 ms Num is : 710 > > > OneBucket Scan cost is : 91 ms Num is : 776 > > > OneBucket Scan cost is : 87 ms Num is : 808 > > > OneBucket Scan cost is : 105 ms Num is : 748 > > > OneBucket Scan cost is : 68 ms Num is : 200 > > > > > > > > > And another question about region split, I want to know which > > RegionServer > > > will load the new region afther splited , > > > Will they be the same One with the old region? Anyone know where I can > > > find the code to learn about that? > > > > > > > > > Thanks for your help > > > > > >
Scan problem
Hello everyone I try to do many Scan use RegionScanner in coprocessor, and ervery time ,the first Scan cost about 10 times than the other, I don't know why this will happen OneBucket Scan cost is : 8794 ms Num is : 710 OneBucket Scan cost is : 91 ms Num is : 776 OneBucket Scan cost is : 87 ms Num is : 808 OneBucket Scan cost is : 105 ms Num is : 748 OneBucket Scan cost is : 68 ms Num is : 200 And another question about region split, I want to know which RegionServer will load the new region afther splited , Will they be the same One with the old region? Anyone know where I can find the code to learn about that? Thanks for your help
Re: WrongRegionException
Both are the same question.
I want to prevent the incoming puts and then copy it as one or more new
puts with different rowkey.
So when there is only one region, my rowkey will belong to it. But when
region splited, some rowkeys may not belong to the new region.
I used to thought HBase will stop new coming puts, finish all of the puts
in the batch, and then try to split.
But this maybe not right according to the exception that I got.
BTY , It seems that I can't add put
to MiniBatchOperationInProgress miniBatchOp. There are only some
functions for get.
Thank you very much for your help
2018-01-29 18:46 GMT+08:00 Anoop John <anoop.hb...@gmail.com>:
> Another related Q was also there.. Can you tell the actual
> requirement? So the incoming puts you want to change the RKs of that?
> Or you want to insert those as well as some new cells with a changed
> RK?
>
> -Anoop-
>
> On Mon, Jan 29, 2018 at 3:49 PM, Yang Zhang <zhang.yang...@gmail.com>
> wrote:
> > Hello Everyone
> >
> > I am using coprocesser to prevent the normal put and replace it
> > with another rowkey, The method is HRegion.put(). It works fine, but when
> > the region splited, There will be an WrongRegionException.
> >
> > 2018-01-28 09:32:51,528 WARN
> > [B.DefaultRpcServer.handler=21,queue=3,port=60020] regionserver.HRegion:
> > Failed getting lock in batch put,
> > row=\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r
> > org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row
> > out of range for row lock on HRegion
> > GISdoop_GeoKey,,1517085124215.341534e84727245f1c67f345c3e467ac.,
> > startKey='', getEndKey()='\xE6G8\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > row='\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r'
> > at org.apache.hadoop.hbase.regionserver.HRegion.checkRow(
> HRegion.java:4677)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.
> getRowLock(HRegion.java:4695)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.
> doMiniBatchMutation(HRegion.java:2786)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.
> batchMutate(HRegion.java:2653)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.
> batchMutate(HRegion.java:2589)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:
> 3192)
> > at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:2459)
> > at site.luoyu.Core.Index.JavaTreeMap.insertPoint(JavaTreeMap.java:287)
> > at site.luoyu.Core.Index.JavaTreeMap.insertRecord(JavaTreeMap.java:256)
> > at site.luoyu.Core.Observer.IndexCopressor.prePut(
> IndexCopressor.java:130)
> > at
> > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(
> RegionCoprocessorHost.java:1122)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.
> java:2674)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.
> batchMutate(HRegion.java:2649)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.
> batchMutate(HRegion.java:2589)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegion.
> batchMutate(HRegion.java:2593)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegionServer.
> doBatchOp(HRegionServer.java:4402)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegionServer.
> doNonAtomicRegionMutation(HRegionServer.java:3584)
> > at
> > org.apache.hadoop.hbase.regionserver.HRegionServer.
> multi(HRegionServer.java:3474)
> > at
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.
> callBlockingMethod(ClientProtos.java:3)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078)
> > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> > at
> > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(
> RpcExecutor.java:114)
> > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > It is said rowkey is out of region's bound. This Exception is logged in
> > regionserver's log as an warning , I can't catch and handle it.
> >
> > According the source code,
> > RowLock rowLock = null;
> > try {
> > rowLock = getRowLock(mutation.getRow(), shouldBlock);
> > } catch (IOException ioe) {
> > LOG.warn("Failed getting lock in batch put, row="
> > + Bytes.toStringBinary(mutation.getRow()), ioe);
> > }
> >
> > HBase just cache and log this exception , I guess it even didn't remove
> it
> > from the batch. So I got so many Exception log and can't put data
> anymore.
> >
> > Why HBase handle this WrongRegionException like this? Anyone can help?
> > Thanks verymuch.
>
WrongRegionException
Hello Everyone
I am using coprocesser to prevent the normal put and replace it
with another rowkey, The method is HRegion.put(). It works fine, but when
the region splited, There will be an WrongRegionException.
2018-01-28 09:32:51,528 WARN
[B.DefaultRpcServer.handler=21,queue=3,port=60020] regionserver.HRegion:
Failed getting lock in batch put,
row=\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r
org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row
out of range for row lock on HRegion
GISdoop_GeoKey,,1517085124215.341534e84727245f1c67f345c3e467ac.,
startKey='', getEndKey()='\xE6G8\x00\x00\x00\x00\x00\x00\x00\x00\x00',
row='\xF0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0F\x10\xC5r'
at org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:4677)
at
org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:4695)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2786)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2653)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2589)
at
org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:3192)
at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:2459)
at site.luoyu.Core.Index.JavaTreeMap.insertPoint(JavaTreeMap.java:287)
at site.luoyu.Core.Index.JavaTreeMap.insertRecord(JavaTreeMap.java:256)
at site.luoyu.Core.Observer.IndexCopressor.prePut(IndexCopressor.java:130)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:1122)
at
org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2674)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2649)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2589)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2593)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4402)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3584)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3474)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:3)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:745)
It is said rowkey is out of region's bound. This Exception is logged in
regionserver's log as an warning , I can't catch and handle it.
According the source code,
RowLock rowLock = null;
try {
rowLock = getRowLock(mutation.getRow(), shouldBlock);
} catch (IOException ioe) {
LOG.warn("Failed getting lock in batch put, row="
+ Bytes.toStringBinary(mutation.getRow()), ioe);
}
HBase just cache and log this exception , I guess it even didn't remove it
from the batch. So I got so many Exception log and can't put data anymore.
Why HBase handle this WrongRegionException like this? Anyone can help?
Thanks verymuch.
Coprocessor prePut
Hi I overrode coprocessor prePut() function to prevent normal put, and try to put the same data with different rowkey into the same table. The code is "context.getEnvironment().getRegion().put(newPut);" . This will caused an exception when the context's region splited. There are so less document about coprocessor, so I write to the mail list .The exception is below. org.apache.hadoop.hbase.NotServingRegionException: GISdoop_GeoKey,\xE6\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01|\xCA\xAD,1516949621636.9628785cfce941207f5d21bdf1eb0f40. is closing at org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5848) at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2249) at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2211) at org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:2787) at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:2095) at site.luoyu.Core.Index.JavaTreeMap.insertPoint(JavaTreeMap.java:286) at site.luoyu.Core.Index.JavaTreeMap.insertRecord(JavaTreeMap.java:257) at site.luoyu.Core.Observer.IndexCopressor.prePut(IndexCopressor.java:130) at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:1122) at org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2280) at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2255) at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2211) at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2215) at org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4283) at org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3490) at org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3380) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29503) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) at java.lang.Thread.run(Thread.java:745) Thanks If anyone can help .
Re: Coprocessor prePut
In fact, I prevent normal writes, and put the same data with a different
row key(Except row key, they have the same columns and same data).
I have already solved the infinite call by add and check some flag
attribute to Put. If the flag exist then skip my code.
This will prevent the infinite call of prePut(). But I want to know If
there are any method support by coprocessor context to do that.
Just like that you can call context.complete() to skip other coprocessors.
Thanks for your advice
2018-01-23 13:01 GMT+08:00 Ted Yu <yuzhih...@gmail.com>:
> Your prePut would write to a different column in the table, right ?
> Otherwise do you prevent normal writes from getting into the same column(s)
> ?
>
> If your prePut writes to dedicated column(s), you can check the presence of
> such column(s) so that the call is not infinite.
>
> Cheers
>
> On Mon, Jan 22, 2018 at 8:56 PM, Yang Zhang <zhang.yang...@gmail.com>
> wrote:
>
> > Yes, It is the same table.
> >
> > 2018-01-23 1:46 GMT+08:00 Ted Yu <yuzhih...@gmail.com>:
> >
> > > Can you clarify your use case ?
> > >
> > > bq. put a data into table
> > >
> > > Does your coprocessor write to the same table which receives user data
> ?
> > >
> > > Cheers
> > >
> > > On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <zhang.yang...@gmail.com>
> > > wrote:
> > >
> > > > Hello Everyone
> > > >
> > > > I am using the coprocessor and want to put another data when
> > > > someone put a data into table. So I try prePut(), but when you call
> > > > HRegion.put(), this will call my coprocessor's prePut function again,
> > and
> > > > it will cause an dead loop.
> > > >
> > > > My code looks like below, If anyone know how to make HBase
> > call
> > > my
> > > > coprocessor just for the first time?
> > > >
> > > > @Override
> > > > public void prePut(ObserverContext
> > > > context, Put put, WALEdit edit,
> > > >Durability durability) throws IOException {
> > > > logger.info(" Coprocessor shouldComplete :
> > > > "+context.shouldComplete()+"");
> > > > context.bypass();
> > > >
> > > > Put myPut = new Put(new RowKey(1).toBytes());
> > > >
> > > > //this call will caused an dead loop
> > > > context.getEnvironment().getRegion().put(put);
> > > >
> > > > }
> > > >
> > > > Thanks anyway
> > > >
> > >
> >
>
Re: Coprocessor prePut
Yes, It is the same table.
2018-01-23 1:46 GMT+08:00 Ted Yu <yuzhih...@gmail.com>:
> Can you clarify your use case ?
>
> bq. put a data into table
>
> Does your coprocessor write to the same table which receives user data ?
>
> Cheers
>
> On Mon, Jan 22, 2018 at 4:24 AM, Yang Zhang <zhang.yang...@gmail.com>
> wrote:
>
> > Hello Everyone
> >
> > I am using the coprocessor and want to put another data when
> > someone put a data into table. So I try prePut(), but when you call
> > HRegion.put(), this will call my coprocessor's prePut function again, and
> > it will cause an dead loop.
> >
> > My code looks like below, If anyone know how to make HBase call
> my
> > coprocessor just for the first time?
> >
> > @Override
> > public void prePut(ObserverContext
> > context, Put put, WALEdit edit,
> >Durability durability) throws IOException {
> > logger.info(" Coprocessor shouldComplete :
> > "+context.shouldComplete()+"");
> > context.bypass();
> >
> > Put myPut = new Put(new RowKey(1).toBytes());
> >
> > //this call will caused an dead loop
> > context.getEnvironment().getRegion().put(put);
> >
> > }
> >
> > Thanks anyway
> >
>
Coprocessor prePut
Hello Everyone
I am using the coprocessor and want to put another data when
someone put a data into table. So I try prePut(), but when you call
HRegion.put(), this will call my coprocessor's prePut function again, and
it will cause an dead loop.
My code looks like below, If anyone know how to make HBase call my
coprocessor just for the first time?
@Override
public void prePut(ObserverContext
context, Put put, WALEdit edit,
Durability durability) throws IOException {
logger.info(" Coprocessor shouldComplete :
"+context.shouldComplete()+"");
context.bypass();
Put myPut = new Put(new RowKey(1).toBytes());
//this call will caused an dead loop
context.getEnvironment().getRegion().put(put);
}
Thanks anyway
Re: Scan Sequence
The suggest range may be larger than end row in this region (For example ,nearby the splited row). So to make sure the result is correct, I need other region to also do the search. I am using zookeeper for lock just for learning whether my theory will work , and how many time it will cost. It's not for release. 2018-01-16 11:34 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > bq. to get an suggest range search to client, and do this suggested range > search > > Since you're using coprocessor, the coprocessor can act (range search) > using the suggestion and return the result to the client. > There is no need to return intermediate result to client. > > BTW using zookeeper for locking is tricky - how do you keep up with region > split / merge ? > > FYI > > On Mon, Jan 15, 2018 at 7:24 PM, Yang Zhang <zhang.yang...@gmail.com> > wrote: > > > In fact , I am using coprocessor to Implement geography index. > Each > > region will only have part of the Index to index the data that it > contained > > . When I try to implement KNN search, only one region will contain the > > index according to the KNN center point. So this region will give an > > suggest of range search to other region（Because other region may also > > contain part of the data in the range search）. > > > > I am using zookeeper to implement the lock, other region will > watch > > until they get the suggest. So if region1 is locked and wait for suggest > > ,and by the time region2 will never get the sacn request and never give > an > > suggest, this caused an dead lock. > > > > In other way, I can search region2 first (by limit the start and > > end) to get an suggest range search to client, and do this suggested > range > > search at the second time. This will work, but It will cost more time. > > > > Any suggest? > > > > Thanks > > > > 2018-01-15 23:49 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > > > > > bq. need region2 to search fist, because it has my own index. > > > > > > Looks like only some of the regions have your index. Can you tell us > more > > > about the characteristics of the region(s) where index would be > present. > > > > > > bq. My scan will be blocked for my lock on region1, > > > > > > By 'my' I assume the lock is placed by your (application) code. Since > > hbase > > > wouldn't know that a subsequent region has index (or suggestion) on > what > > to > > > scan. > > > Can you give us more detail ? > > > > > > Thanks > > > > > > On Mon, Jan 15, 2018 at 7:42 AM, Yang Zhang <zhang.yang...@gmail.com> > > > wrote: > > > > > > > Hello Everyone > > > > > > > > I want to know If the Scan is send to HBase by sequence? For > > > > example, If I have a table that has two region（region1,region2）, and > I > > > sent > > > > a Scan() to this Table. This Scan will be send to region1 first and > > until > > > > region1 returned, region2 can get this Scan? So If There are some > > methods > > > > than I can control the sequence? > > > > > > > > some time I need region2 to search fist, because it has my own > > > index. > > > > Then It will give an suggest to other region about where to search.. > I > > am > > > > using zookeeper to do this, But I found that My scan will be blocked > > for > > > my > > > > lock on region1, and region2 who has the suggest, even't can't > receive > > > this > > > > scan. > > > > > > > > Anyone have suggest? > > > > > > > > > > > > Thanks very much > > > > > > > > > >
Re: Scan Sequence
In fact , I am using coprocessor to Implement geography index. Each region will only have part of the Index to index the data that it contained . When I try to implement KNN search, only one region will contain the index according to the KNN center point. So this region will give an suggest of range search to other region（Because other region may also contain part of the data in the range search）. I am using zookeeper to implement the lock, other region will watch until they get the suggest. So if region1 is locked and wait for suggest ,and by the time region2 will never get the sacn request and never give an suggest, this caused an dead lock. In other way, I can search region2 first (by limit the start and end) to get an suggest range search to client, and do this suggested range search at the second time. This will work, but It will cost more time. Any suggest? Thanks 2018-01-15 23:49 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > bq. need region2 to search fist, because it has my own index. > > Looks like only some of the regions have your index. Can you tell us more > about the characteristics of the region(s) where index would be present. > > bq. My scan will be blocked for my lock on region1, > > By 'my' I assume the lock is placed by your (application) code. Since hbase > wouldn't know that a subsequent region has index (or suggestion) on what to > scan. > Can you give us more detail ? > > Thanks > > On Mon, Jan 15, 2018 at 7:42 AM, Yang Zhang <zhang.yang...@gmail.com> > wrote: > > > Hello Everyone > > > > I want to know If the Scan is send to HBase by sequence? For > > example, If I have a table that has two region（region1,region2）, and I > sent > > a Scan() to this Table. This Scan will be send to region1 first and until > > region1 returned, region2 can get this Scan? So If There are some methods > > than I can control the sequence? > > > > some time I need region2 to search fist, because it has my own > index. > > Then It will give an suggest to other region about where to search.. I am > > using zookeeper to do this, But I found that My scan will be blocked for > my > > lock on region1, and region2 who has the suggest, even't can't receive > this > > scan. > > > > Anyone have suggest? > > > > > > Thanks very much > > >
Scan Sequence
Hello Everyone I want to know If the Scan is send to HBase by sequence? For example, If I have a table that has two region（region1,region2）, and I sent a Scan() to this Table. This Scan will be send to region1 first and until region1 returned, region2 can get this Scan? So If There are some methods than I can control the sequence? some time I need region2 to search fist, because it has my own index. Then It will give an suggest to other region about where to search.. I am using zookeeper to do this, But I found that My scan will be blocked for my lock on region1, and region2 who has the suggest, even't can't receive this scan. Anyone have suggest? Thanks very much
Question About preSplit() hook
Hello everyone I am using coprocessor of HBase, My work is to split my own index when RegionSplit. When I use the preSplit(ObserverContext c, byte[] splitRow) , I thought The param splitRow must be some row taht I puted in my Table, But this time I get one row That can't be mine(my rowkey is constructed with long+int but the splitRow is only 3bytes) So I want to know how HBase determine the splitRow? Is the splitRow must be exist in my table? If anybody can help? Thanks very much!
Re: Some problem about RegionScanner In coprocessor
Thank you very much, It works!, and sorry for my stupid code.
2017-12-10 18:26 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>:
> You need to clear the values every time before you call rs.next otherwise
> the list will become larger and larger.
>
> 2017-12-10 15:05 GMT+08:00 Yang Zhang <zhang.yang...@gmail.com>:
>
> > Hello everyone
> >
> > I am using coprocessor in HBase. When I try to Scan a Region
> using
> > RegionScanner , It will takes a long time. Even 100 times longer than
> Scan
> > from HBase shell. So I believe there must be something wrong.
> >
> > What's more, HBase will also give no response when I try to
> disable
> > this table and try to delete.
> >
> > My HBase version is 0.98.4,,Table row number is 50,all data
> is
> > in one region.I do this in preScannerOpen hooks.
> >
> > My code is below,Anybody can help? Thanks very much!
> >
> > private void testScan(String regionInfo, RegionCoprocessorEnvironment
> > env,RegionScanner regionScanner){
> > IndexTree indexTree = new JavaTreeMap(new Node());
> > regionIndexMap.put(regionInfo, indexTree);
> > Scan scan = new Scan();
> > scan.setFilter(new FirstKeyOnlyFilter());
> >
> > long totalNum = 0;
> > long start = System.currentTimeMillis();
> >
> > try {
> > RegionScanner rs = env.getRegion().getScanner(scan);
> > List values = new LinkedList<>();
> > while (rs.next(values)){
> > totalNum++;
> > }
> > } catch (IOException e) {
> > e.printStackTrace();
> > }
> > long end = System.currentTimeMillis();
> > logger.error(" TestScanAllRegion Cost :"+(end-start)+"
> > RegionNum :"+totalNum);
> > }
> >
>
Some problem about RegionScanner In coprocessor
Hello everyone
I am using coprocessor in HBase. When I try to Scan a Region using
RegionScanner , It will takes a long time. Even 100 times longer than Scan
from HBase shell. So I believe there must be something wrong.
What's more, HBase will also give no response when I try to disable
this table and try to delete.
My HBase version is 0.98.4,,Table row number is 50,all data is
in one region.I do this in preScannerOpen hooks.
My code is below,Anybody can help? Thanks very much!
private void testScan(String regionInfo, RegionCoprocessorEnvironment
env,RegionScanner regionScanner){
IndexTree indexTree = new JavaTreeMap(new Node());
regionIndexMap.put(regionInfo, indexTree);
Scan scan = new Scan();
scan.setFilter(new FirstKeyOnlyFilter());
long totalNum = 0;
long start = System.currentTimeMillis();
try {
RegionScanner rs = env.getRegion().getScanner(scan);
List values = new LinkedList<>();
while (rs.next(values)){
totalNum++;
}
} catch (IOException e) {
e.printStackTrace();
}
long end = System.currentTimeMillis();
logger.error(" TestScanAllRegion Cost :"+(end-start)+"
RegionNum :"+totalNum);
}
Re: How to Updates coprocessor
Thanks，I have already read that.I Just confuse whether the JVM will unload my coprocessor class and load the new one? According to the article , they also said like This: *Updating a Coprocessor* *Deploying a new version of a given coprocessor is not as simple as disabling it, replacing the JAR, and re-enabling the coprocessor. This is because you cannot reload a class in a JVM unless you delete all the current references to it. Since the current JVM has reference to the existing coprocessor, you must restart the JVM, by restarting the RegionServer, in order to replace it. This behavior is not expected to change.* So if I want to unload the old coprocessor, I need to restart JVM by restart HBase? 2017-07-09 9:40 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > Have you read http://hbase.apache.org/book.html#load_coprocessor_in_shell > ? > > table_att_unset should be used in the alter command. > > The last command should be: > > enable 'mytable' > > When the regions of mytable reopen, they should load updated coprocessor. > > On Sat, Jul 8, 2017 at 6:35 PM, Yang Zhang <zhang.yang...@gmail.com> > wrote: > > > When I develop my program with coprocessor. I upload and dynamic install > > my coprocessor. > > But when I want to do update my coprocessor, I do these thing below: > > > > > disable 'mytable' > > > METHOD => 'table_att_unset',NAME => 'coprocessor$1' > > > alter 'mytable',METHOD=>'table_att','coprocessor'=>'hdfs:/// > > GISdoop.jar|site.luoyu.MyFirstCop|1001' > > > enable 'Coprocessor' > > > > Will this unload The class from Classloader and load my new class right > > way? > > >
How to Updates coprocessor
When I develop my program with coprocessor. I upload and dynamic install my coprocessor. But when I want to do update my coprocessor, I do these thing below: > disable 'mytable' > METHOD => 'table_att_unset',NAME => 'coprocessor$1' > alter 'mytable',METHOD=>'table_att','coprocessor'=>'hdfs:/// GISdoop.jar|site.luoyu.MyFirstCop|1001' > enable 'Coprocessor' Will this unload The class from Classloader and load my new class right way?
