Because internally deleteColumn with LATEST_TIMESTAMP does a get to find
the exact latest version on which the delete applies we do a get. User
perspective it may not be needed but since a read happens on those blocks
where the latest version of the cell resides we have to know the metric.
Even the block may get loaded to the block cache if it was not already
loaded.
I think it is ok to have this.
Regards
Ram
On Fri, May 23, 2014 at 7:14 AM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Hi Vinay,
>
> > But regarding my earlier concern, calling get in case of delete request,
> in order to check the time stamp, can we not query hbase directly using the
> scanner avoiding the CP hooks..??
>
> If you configure a CP to be called, HBase will call it. There isn't really
> any way to avoid that. You can maybe modify your CPs to track the details
> of the gets, or something like that, do identify if it's a get before a
> delete or not. But still, I'm wondering why you are using deleteColumn and
> not deleteColumns.
>
> > Also I observe in the code that read metrics are updated in the
> org.apache.hadoop.hbase.regionserver.HRegion.get(Get, boolean) as part of
> preDeleteTimeStamps call. I feel this delete request affect the read
> metrics as well. Why should the read metrics be updated by a delete
> request..??
> If delete calls get to read something, there there is a read activity, and
> this need to be monitored in the metric side. Again, calling deleteColumns
> will avoid this get and so will avoid this metrics update ;)
>
> JM
>
>
> 2014-05-21 1:31 GMT-04:00 Vinay Kashyap <vinay_kash...@ymail.com>:
>
> > Hi Jean,
> >
> > I rechecked the logic regarding delete with specifying only 'key', there
> > was a small mistake in the condition, I have corrected the same and now
> > getting the CP hook. Thanks for that.
> >
> > But regarding my earlier concern, calling get in case of delete request,
> > in order to check the time stamp, can we not query hbase directly using
> the
> > scanner avoiding the CP hooks..??
> >
> > Also I observe in the code that read metrics are updated in the
> > org.apache.hadoop.hbase.regionserver.HRegion.get(Get, boolean) as part of
> > preDeleteTimeStamps call. I feel this delete request affect the read
> > metrics as well. Why should the read metrics be updated by a delete
> > request..??
> >
> >
> >
> > Thanks
> > Vinay Kashyap
> >
> >
> > On Tuesday, 20 May 2014 8:53 PM, Jean-Marc Spaggiari <
> > jean-m...@spaggiari.org> wrote:
> >
> >
> >
> > Hi Vinay,
> >
> > I don't see anything wrong in the code which will not trigger the
> preDelete
> > hook when you do a delete with only the row key. It should be identical
> as
> > adding all the families.
> >
> > Server side, this is the piece of code called when you are doing a
> > Delete(rowkey) with nothing else:
> >
> > void prepareDelete(Delete delete) throws IOException {
> > // Check to see if this is a deleteRow insert
> > if(delete.getFamilyCellMap().isEmpty()){
> > for(byte [] family : this.htableDescriptor.getFamiliesKeys()){
> > // Don't eat the timestamp
> > delete.deleteFamily(family, delete.getTimeStamp());
> > }
> > }
> > ....
> >
> > as you can see if your don't give anything, HBase will just add all the
> > families. And the rest of the execution is the same as the other path.
> Can
> > you re-run your tests to double check?
> >
> > Thanks,
> >
> > JM
> >
> >
> >
> > 2014-05-20 7:52 GMT-04:00 Jean-Marc Spaggiari <jean-m...@spaggiari.org>:
> >
> > > Hi Vinay,
> > >
> > > Coprocessors promess is to be called whenever there is a get, a
> delete, a
> > > put, etc. So since deleteColumn do a get and a delete, I thinks it's
> > > "normal" to get the 2 methods called.
> > >
> > > If you can not change deleteColumn by deleteColumns, what you can do
> is,
> > > on the delete hook, you can check the timestamp and take note if it's
> > > LATEST_TIMESTAMP because it means it's a deleteColumn call and there
> has
> > > been a call to preGet too.
> > >
> > > deleteColumn is to delete only the last version, so we don't have any
> > > other choice then looking first for what is the last version before
> being
> > > able to delete it.
> > >
> > > Regarding you last point, if that's realy the case, this i snot
> normal. I
> > > will take a look at the code later today (most probably tonight) and
> let
> > > you know.
> > >
> > > JM
> > >
> > >
> > > 2014-05-20 7:08 GMT-04:00 Vinay Kashyap <vinay_kash...@ymail.com>:
> > >
> > > Hi Jean,
> > >>
> > >> Thanks for your detailed info.
> > >> Now I understand the trace from where the CP hook is getting called.
> > >> But with this behavior of the delete request, don't you think there
> is a
> > >> kind of restriction to the delete request by the client.??
> > >> By restriction I meant, for an application which collects some
> > statistics
> > >> using of the CP hooks, will vary according to the type of request
> from
> > the
> > >> Client. What is your opinion about this..??
> > >>
> > >>
> > >> Also, I noticed another thing about Delete request. When the client
> > >> request for delete by specifying only 'key' and not any specifying any
> > >> columns [ same as deleteAll from hbase shell ] , it does not give any
> > call
> > >> to the CP. Any way I can get a CP hook for such delete requests.??
> > >>
> > >>
> > >> Thanks
> > >> Vinay Kashyap
> > >>
> > >>
> > >> On Monday, 19 May 2014 7:12 PM, Jean-Marc Spaggiari <
> > >> jean-m...@spaggiari.org> wrote:
> > >>
> > >>
> > >>
> > >> Hi Vinay,
> > >>
> > >> In 0.94. When you call the deleteColumn:
> > >> public Delete deleteColumn(byte [] family, byte [] qualifier) {
> > >> this.deleteColumn(family, qualifier, HConstants.LATEST_TIMESTAMP);
> > >> return this;
> > >> }
> > >>
> > >>
> > >> You set the timestamp to LATEST_TIMESTAMP. Then on the server side, in
> > >> prepareDeleteTimestamps, isLatestTimestamp verify if it's
> > LATEST_TIMESTAMP
> > >> and if so to the following get:
> > >> Get get = new Get(kv.getRow());
> > >> get.setMaxVersions(count);
> > >> get.addColumn(family, qual);
> > >>
> > >> List<KeyValue> result = get(get, false);
> > >>
> > >> if (result.size() < count) {
> > >> // Nothing to delete
> > >> kv.updateLatestStamp(byteNow);
> > >> continue;
> > >> }
> > >>
> > >> In trunk it's very similar except that default ts is LATEST_TIMESTAMP
> so
> > >> the call is the following:
> > >> public Delete deleteColumn(byte [] family, byte [] qualifier) {
> > >> this.deleteColumn(family, qualifier, this.ts);
> > >> return this;
> > >> }
> > >> So it you don't change the timestamp manually when you create the
> delete
> > >> object, ts still is LATEST_TIMESTAMP and the behavious in HRegion is
> the
> > >> same as above.
> > >>
> > >> In the "get(get, false)" call you will see the call to the CP hook:
> > >> // pre-get CP hook
> > >> if (withCoprocessor && (coprocessorHost != null)) {
> > >> if (coprocessorHost.preGet(get, results)) {
> > >> return results;
> > >> }
> > >> }
> > >>
> > >>
> > >> So. When you call deleteColumn you indirectly call
> > >> coprocessorHost.preGet(get, results).
> > >>
> > >> JM
> > >>
> > >>
> > >>
> > >> 2014-05-19 5:39 GMT-04:00 Vinay Kashyap <vinay_kash...@ymail.com>:
> > >>
> > >> > Hi Jean,
> > >> >
> > >> > Thanks for your information.
> > >> > I am using deleteColumn in my application. I will check the behavior
> > >> once
> > >> > by changing it to use deleteColumns as you suggested.
> > >> > But is there any difference in the CP hooks for a delete request.??
> > >> > Because, in my CP I have implemented preDelete() and in order to
> keep
> > >> my CP
> > >> > generic for any client request, is there any other hook on the CP so
> > >> that I
> > >> > can process the delete requests accordingly.??
> > >> > Also, can you tell me the exact class file which has this logic to
> > check
> > >> > for the time stamp for a delete request.??
> > >> >
> > >> >
> > >> > Thanks
> > >> > Vinay Kashyap
> > >> > On Saturday, 17 May 2014 11:15 PM, Jean-Marc Spaggiari <
> > >> > jean-m...@spaggiari.org> wrote:
> > >> >
> > >> >
> > >> >
> > >> > Thanks for the confirmatoin Ted ;) I figured that afterward that
> some
> > >> > emails are not coming in the right order.
> > >> >
> > >> > Vinay, can you please confirm the client call you are doing?
> > >> >
> > >> > Thanks,
> > >> >
> > >> > JM
> > >> >
> > >> >
> > >> >
> > >> > 2014-05-16 19:19 GMT-04:00 Ted Yu <yuzhih...@gmail.com>:
> > >> >
> > >> > > JMS:
> > >> > > I saw your earlier email.
> > >> > >
> > >> > > There are some emails in ASF queue being processed (backlog).
> > >> > >
> > >> > > FYI
> > >> > >
> > >> > > On Fri, May 16, 2014 at 5:05 AM, Jean-Marc Spaggiari <
> > >> > > jean-m...@spaggiari.org> wrote:
> > >> > >
> > >> > > > Did anyone received the email I sent below? Based on the other
> > >> emails,
> > >> > > > seems not. So just re-sending.
> > >> > > >
> > >> > > > JM
> > >> > > >
> > >> > > >
> > >> > > > 2014-05-15 8:24 GMT-04:00 Jean-Marc Spaggiari <
> > >> jean-m...@spaggiari.org
> > >> > >:
> > >> > > >
> > >> > > > > Hi Vinay,
> > >> > > > >
> > >> > > > > If you use deleteColumn (with no S), HBase need to first to a
> > get
> > >> to
> > >> > > find
> > >> > > > > the last timeStamp, then do the delete. If you goal is to
> remove
> > >> all
> > >> > > the
> > >> > > > > versions of the specific column, you should use deleteColumns,
> > >> which
> > >> > > will
> > >> > > > > not all get.
> > >> > > > >
> > >> > > > > JM
> > >> > > > >
> > >> > > > >
> > >> > > > > 2014-05-13 10:16 GMT-04:00 Vinay Kashyap <
> > vinay_kash...@ymail.com
> > >> >:
> > >> > > > >
> > >> > > > > Dear all,
> > >> > > > >>
> > >> > > > >> I am using HBase 0.96.1.1-hadoop2 with CDH-5.0.0.
> > >> > > > >> I have an application where I have registered a coprocessor
> to
> > my
> > >> > > table
> > >> > > > >> to get few statistics on the read/write/delete requests.
> > >> > > > >> I have implemented preGetOp, prePut and preDelete accordingly
> > >> and it
> > >> > > is
> > >> > > > >> working as expected in case of read/write requests.
> > >> > > > >> But when I issue a delete request on the table, coprocessor's
> > >> > preGetOp
> > >> > > > is
> > >> > > > >> been called which is varying the read requests statistics.
> > >> > > > >> I wanted to understand why is the preGetOp being called when
> > >> delete
> > >> > > > >> request is issued.?
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> Thanks and regards
> > >> > > > >> Vinay Kashyap
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
