Hi Jean,
I rechecked the logic regarding delete with specifying only 'key', there was a
small mistake in the condition, I have corrected the same and now getting the
CP hook. Thanks for that.
But regarding my earlier concern, calling get in case of delete request, in
order to check the time stamp, can we not query hbase directly using the
scanner avoiding the CP hooks..??
Also I observe in the code that read metrics are updated in the
org.apache.hadoop.hbase.regionserver.HRegion.get(Get, boolean) as part of
preDeleteTimeStamps call. I feel this delete request affect the read metrics as
well. Why should the read metrics be updated by a delete request..??
Thanks
Vinay Kashyap
On Tuesday, 20 May 2014 8:53 PM, Jean-Marc Spaggiari <jean-m...@spaggiari.org>
wrote:
Hi Vinay,
I don't see anything wrong in the code which will not trigger the preDelete
hook when you do a delete with only the row key. It should be identical as
adding all the families.
Server side, this is the piece of code called when you are doing a
Delete(rowkey) with nothing else:
void prepareDelete(Delete delete) throws IOException {
// Check to see if this is a deleteRow insert
if(delete.getFamilyCellMap().isEmpty()){
for(byte [] family : this.htableDescriptor.getFamiliesKeys()){
// Don't eat the timestamp
delete.deleteFamily(family, delete.getTimeStamp());
}
}
....
as you can see if your don't give anything, HBase will just add all the
families. And the rest of the execution is the same as the other path. Can
you re-run your tests to double check?
Thanks,
JM
2014-05-20 7:52 GMT-04:00 Jean-Marc Spaggiari <jean-m...@spaggiari.org>:
> Hi Vinay,
>
> Coprocessors promess is to be called whenever there is a get, a delete, a
> put, etc. So since deleteColumn do a get and a delete, I thinks it's
> "normal" to get the 2 methods called.
>
> If you can not change deleteColumn by deleteColumns, what you can do is,
> on the delete hook, you can check the timestamp and take note if it's
> LATEST_TIMESTAMP because it means it's a deleteColumn call and there has
> been a call to preGet too.
>
> deleteColumn is to delete only the last version, so we don't have any
> other choice then looking first for what is the last version before being
> able to delete it.
>
> Regarding you last point, if that's realy the case, this i snot normal. I
> will take a look at the code later today (most probably tonight) and let
> you know.
>
> JM
>
>
> 2014-05-20 7:08 GMT-04:00 Vinay Kashyap <vinay_kash...@ymail.com>:
>
> Hi Jean,
>>
>> Thanks for your detailed info.
>> Now I understand the trace from where the CP hook is getting called.
>> But with this behavior of the delete request, don't you think there is a
>> kind of restriction to the delete request by the client.??
>> By restriction I meant, for an application which collects some statistics
>> using of the CP hooks, will vary according to the type of request from the
>> Client. What is your opinion about this..??
>>
>>
>> Also, I noticed another thing about Delete request. When the client
>> request for delete by specifying only 'key' and not any specifying any
>> columns [ same as deleteAll from hbase shell ] , it does not give any call
>> to the CP. Any way I can get a CP hook for such delete requests.??
>>
>>
>> Thanks
>> Vinay Kashyap
>>
>>
>> On Monday, 19 May 2014 7:12 PM, Jean-Marc Spaggiari <
>> jean-m...@spaggiari.org> wrote:
>>
>>
>>
>> Hi Vinay,
>>
>> In 0.94. When you call the deleteColumn:
>> public Delete deleteColumn(byte [] family, byte [] qualifier) {
>> this.deleteColumn(family, qualifier, HConstants.LATEST_TIMESTAMP);
>> return this;
>> }
>>
>>
>> You set the timestamp to LATEST_TIMESTAMP. Then on the server side, in
>> prepareDeleteTimestamps, isLatestTimestamp verify if it's LATEST_TIMESTAMP
>> and if so to the following get:
>> Get get = new Get(kv.getRow());
>> get.setMaxVersions(count);
>> get.addColumn(family, qual);
>>
>> List<KeyValue> result = get(get, false);
>>
>> if (result.size() < count) {
>> // Nothing to delete
>> kv.updateLatestStamp(byteNow);
>> continue;
>> }
>>
>> In trunk it's very similar except that default ts is LATEST_TIMESTAMP so
>> the call is the following:
>> public Delete deleteColumn(byte [] family, byte [] qualifier) {
>> this.deleteColumn(family, qualifier, this.ts);
>> return this;
>> }
>> So it you don't change the timestamp manually when you create the delete
>> object, ts still is LATEST_TIMESTAMP and the behavious in HRegion is the
>> same as above.
>>
>> In the "get(get, false)" call you will see the call to the CP hook:
>> // pre-get CP hook
>> if (withCoprocessor && (coprocessorHost != null)) {
>> if (coprocessorHost.preGet(get, results)) {
>> return results;
>> }
>> }
>>
>>
>> So. When you call deleteColumn you indirectly call
>> coprocessorHost.preGet(get, results).
>>
>> JM
>>
>>
>>
>> 2014-05-19 5:39 GMT-04:00 Vinay Kashyap <vinay_kash...@ymail.com>:
>>
>> > Hi Jean,
>> >
>> > Thanks for your information.
>> > I am using deleteColumn in my application. I will check the behavior
>> once
>> > by changing it to use deleteColumns as you suggested.
>> > But is there any difference in the CP hooks for a delete request.??
>> > Because, in my CP I have implemented preDelete() and in order to keep
>> my CP
>> > generic for any client request, is there any other hook on the CP so
>> that I
>> > can process the delete requests accordingly.??
>> > Also, can you tell me the exact class file which has this logic to check
>> > for the time stamp for a delete request.??
>> >
>> >
>> > Thanks
>> > Vinay Kashyap
>> > On Saturday, 17 May 2014 11:15 PM, Jean-Marc Spaggiari <
>> > jean-m...@spaggiari.org> wrote:
>> >
>> >
>> >
>> > Thanks for the confirmatoin Ted ;) I figured that afterward that some
>> > emails are not coming in the right order.
>> >
>> > Vinay, can you please confirm the client call you are doing?
>> >
>> > Thanks,
>> >
>> > JM
>> >
>> >
>> >
>> > 2014-05-16 19:19 GMT-04:00 Ted Yu <yuzhih...@gmail.com>:
>> >
>> > > JMS:
>> > > I saw your earlier email.
>> > >
>> > > There are some emails in ASF queue being processed (backlog).
>> > >
>> > > FYI
>> > >
>> > > On Fri, May 16, 2014 at 5:05 AM, Jean-Marc Spaggiari <
>> > > jean-m...@spaggiari.org> wrote:
>> > >
>> > > > Did anyone received the email I sent below? Based on the other
>> emails,
>> > > > seems not. So just re-sending.
>> > > >
>> > > > JM
>> > > >
>> > > >
>> > > > 2014-05-15 8:24 GMT-04:00 Jean-Marc Spaggiari <
>> jean-m...@spaggiari.org
>> > >:
>> > > >
>> > > > > Hi Vinay,
>> > > > >
>> > > > > If you use deleteColumn (with no S), HBase need to first to a get
>> to
>> > > find
>> > > > > the last timeStamp, then do the delete. If you goal is to remove
>> all
>> > > the
>> > > > > versions of the specific column, you should use deleteColumns,
>> which
>> > > will
>> > > > > not all get.
>> > > > >
>> > > > > JM
>> > > > >
>> > > > >
>> > > > > 2014-05-13 10:16 GMT-04:00 Vinay Kashyap <vinay_kash...@ymail.com
>> >:
>> > > > >
>> > > > > Dear all,
>> > > > >>
>> > > > >> I am using HBase 0.96.1.1-hadoop2 with CDH-5.0.0.
>> > > > >> I have an application where I have registered a coprocessor to my
>> > > table
>> > > > >> to get few statistics on the read/write/delete requests.
>> > > > >> I have implemented preGetOp, prePut and preDelete accordingly
>> and it
>> > > is
>> > > > >> working as expected in case of read/write requests.
>> > > > >> But when I issue a delete request on the table, coprocessor's
>> > preGetOp
>> > > > is
>> > > > >> been called which is varying the read requests statistics.
>> > > > >> I wanted to understand why is the preGetOp being called when
>> delete
>> > > > >> request is issued.?
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> Thanks and regards
>> > > > >> Vinay Kashyap
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
