Re: Incremental import from HBase to Hive

2017-01-28 Thread Rohit Jain
An example of that is how in Trafodion one can generate a Divisioning column, such a week number, derived from a date column, that becomes the leading part of a multi-column HBase key. Of course, Trafodion has a salt key as a prefix to spread the data across the regions in a balanced way, but yo

Re: Incremental import from HBase to Hive

2017-01-28 Thread Josh Elser
(Please stop adding the dev@hbase mailing list. This is a question for the user@ list only.) Unless you have a time component included in your HBase data, there is no way to find all "new" data in HBase with the timestamp component aside from scanning the entire HBase table. Performing a full

Re: Question on WALEdit

2017-01-28 Thread Ted Yu
In CellUtil, there is the following method: public static Tag getTag(Cell cell, byte type){ In MobUtils, you can find sample usage: public static Tag getTableNameTag(Cell cell) { if (cell.getTagsLength() > 0) { return CellUtil.getTag(cell, TagType.MOB_TABLE_NAME_TAG_TYPE); FYI

Re: Writing/Importing large number of records into HBase

2017-01-28 Thread jeff saremi
No iI had not.I will take a look. Thanks Ted From: Ted Yu Sent: Friday, January 27, 2017 7:41 PM To: user@hbase.apache.org Subject: Re: Writing/Importing large number of records into HBase Have you looked at hbase-spark module (currently in master branch) ? See

Re: Writing/Importing large number of records into HBase

2017-01-28 Thread jeff saremi
Thank you Chetan From: Chetan Khatri Sent: Friday, January 27, 2017 8:15 PM To: user@hbase.apache.org Subject: Re: Writing/Importing large number of records into HBase Oh. Sorry. https://github.com/apache/hbase/blob/master/hbase-spark/src/main/java/org/apache/ha

Re: Question on WALEdit

2017-01-28 Thread Ted Yu
I haven't found the API you were looking for. Which release of hbase are you using ? I assume it supports tags. If you use tag to pass event-id, you can retrieve thru this method of WALEdit: public ArrayList getCells() { >From Cell, there're 3 methods for retrieving tag starting with: byte

Question on WALEdit

2017-01-28 Thread Sreeram
Hi, TL;DR: In my use case I am setting attributes for Puts and Deletes using setAttribute(). I would like to know if it is possible to get the attributes that I had set from the WALEdit ? Here is my use case in detail: I have a replicated cluster A which gets replicated to cluster B. From cluste