Re: Custom Filter on hbase Column
Thanks Anoop.I did that but the only method that was getting called in my filter was public byte[] toByteArray() ,even though I over ride transformcell. Thanks, Nishanth On Thu, Sep 11, 2014 at 10:51 PM, Anoop John anoop.hb...@gmail.com wrote: And u have to implement transformCell(*final* Cell v) in your custom Filter. JFYI -Anoop- On Fri, Sep 12, 2014 at 4:36 AM, Nishanth S nishanth.2...@gmail.com wrote: Sure Sean.This is much needed. -Nishan On Thu, Sep 11, 2014 at 3:57 PM, Sean Busbey bus...@cloudera.com wrote: I filed HBASE-11950 to get some details added to the book on this topic[1]. Nishanth, could you follow that ticket and give feedback on whatever update ends up proposed? [1]: https://issues.apache.org/jira/browse/HBASE-11950 On Thu, Sep 11, 2014 at 4:40 PM, Ted Yu yuzhih...@gmail.com wrote: See http://search-hadoop.com/m/DHED4xWh622 On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hey All, I am sorry if this is a naive question.Do we need to generate a proto file using proto buffer compiler when implementing a filter.I did not see that any where in the documentation.Can some one help please? On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan -- Sean
Re: Custom Filter on hbase Column
Take a look at StoreScanner#next(): ScanQueryMatcher.MatchCode qcode = matcher.match(kv); switch(qcode) { case INCLUDE: case INCLUDE_AND_SEEK_NEXT_ROW: case INCLUDE_AND_SEEK_NEXT_COL: Filter f = matcher.getFilter(); if (f != null) { // TODO convert Scan Query Matcher to be Cell instead of KV based ? kv = KeyValueUtil.ensureKeyValue(f.transformCell(kv)); } Cheers On Mon, Sep 15, 2014 at 8:27 AM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Anoop.I did that but the only method that was getting called in my filter was public byte[] toByteArray() ,even though I over ride transformcell. Thanks, Nishanth On Thu, Sep 11, 2014 at 10:51 PM, Anoop John anoop.hb...@gmail.com wrote: And u have to implement transformCell(*final* Cell v) in your custom Filter. JFYI -Anoop- On Fri, Sep 12, 2014 at 4:36 AM, Nishanth S nishanth.2...@gmail.com wrote: Sure Sean.This is much needed. -Nishan On Thu, Sep 11, 2014 at 3:57 PM, Sean Busbey bus...@cloudera.com wrote: I filed HBASE-11950 to get some details added to the book on this topic[1]. Nishanth, could you follow that ticket and give feedback on whatever update ends up proposed? [1]: https://issues.apache.org/jira/browse/HBASE-11950 On Thu, Sep 11, 2014 at 4:40 PM, Ted Yu yuzhih...@gmail.com wrote: See http://search-hadoop.com/m/DHED4xWh622 On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hey All, I am sorry if this is a naive question.Do we need to generate a proto file using proto buffer compiler when implementing a filter.I did not see that any where in the documentation.Can some one help please? On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan -- Sean
Re: Custom Filter on hbase Column
Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan
Re: Custom Filter on hbase Column
Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan
Re: Custom Filter on hbase Column
Hey All, I am sorry if this is a naive question.Do we need to generate a proto file using proto buffer compiler when implementing a filter.I did not see that any where in the documentation.Can some one help please? On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan
Re: Custom Filter on hbase Column
See http://search-hadoop.com/m/DHED4xWh622 On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hey All, I am sorry if this is a naive question.Do we need to generate a proto file using proto buffer compiler when implementing a filter.I did not see that any where in the documentation.Can some one help please? On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan
Re: Custom Filter on hbase Column
I filed HBASE-11950 to get some details added to the book on this topic[1]. Nishanth, could you follow that ticket and give feedback on whatever update ends up proposed? [1]: https://issues.apache.org/jira/browse/HBASE-11950 On Thu, Sep 11, 2014 at 4:40 PM, Ted Yu yuzhih...@gmail.com wrote: See http://search-hadoop.com/m/DHED4xWh622 On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hey All, I am sorry if this is a naive question.Do we need to generate a proto file using proto buffer compiler when implementing a filter.I did not see that any where in the documentation.Can some one help please? On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan -- Sean
Re: Custom Filter on hbase Column
Sure Sean.This is much needed. -Nishan On Thu, Sep 11, 2014 at 3:57 PM, Sean Busbey bus...@cloudera.com wrote: I filed HBASE-11950 to get some details added to the book on this topic[1]. Nishanth, could you follow that ticket and give feedback on whatever update ends up proposed? [1]: https://issues.apache.org/jira/browse/HBASE-11950 On Thu, Sep 11, 2014 at 4:40 PM, Ted Yu yuzhih...@gmail.com wrote: See http://search-hadoop.com/m/DHED4xWh622 On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hey All, I am sorry if this is a naive question.Do we need to generate a proto file using proto buffer compiler when implementing a filter.I did not see that any where in the documentation.Can some one help please? On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan -- Sean
Re: Custom Filter on hbase Column
And u have to implement transformCell(*final* Cell v) in your custom Filter. JFYI -Anoop- On Fri, Sep 12, 2014 at 4:36 AM, Nishanth S nishanth.2...@gmail.com wrote: Sure Sean.This is much needed. -Nishan On Thu, Sep 11, 2014 at 3:57 PM, Sean Busbey bus...@cloudera.com wrote: I filed HBASE-11950 to get some details added to the book on this topic[1]. Nishanth, could you follow that ticket and give feedback on whatever update ends up proposed? [1]: https://issues.apache.org/jira/browse/HBASE-11950 On Thu, Sep 11, 2014 at 4:40 PM, Ted Yu yuzhih...@gmail.com wrote: See http://search-hadoop.com/m/DHED4xWh622 On Thu, Sep 11, 2014 at 2:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hey All, I am sorry if this is a naive question.Do we need to generate a proto file using proto buffer compiler when implementing a filter.I did not see that any where in the documentation.Can some one help please? On Thu, Sep 11, 2014 at 12:41 PM, Nishanth S nishanth.2...@gmail.com wrote: Thanks Dima and Ted.Yes I need to return the first the 1000 characters.There is no matching involved. -Nishan On Thu, Sep 11, 2014 at 12:24 PM, Ted Yu yuzhih...@gmail.com wrote: In Nishanth's case, the 5K message is stored in one KeyValue, right ? If only the first 1000 Characters of this message are to be returned, a new KeyValue needs to be composed and returned. Cheers On Thu, Sep 11, 2014 at 11:09 AM, Dima Spivak dspi...@cloudera.com wrote: Hi Nishanth, Take a look at http://hbase.apache.org/book/client.filter.html . If I understand your use case correctly, you might want to look at RegexStringComparator to match the first 1000 characters of your column qualifier. -Dima On Thu, Sep 11, 2014 at 12:37 PM, Nishanth S nishanth.2...@gmail.com wrote: Hi All, I have an hbase table with multiple cfs (say c1,c2,c3).Each of this column family has a column 'message' which is about 5K.What I need to do is to grab only the first 1000 Characters of this message when I do a get on the table using row Key.I was thinking of using filters to do this on hbase sever side.Can some one help me on how to go about this. Thanks, Nishan -- Sean