Re: How to get Last 1000 records from 1 millions records

2016-08-25 Thread Ted Yu
Proper data structure at client side can avoid sorting. For example: https://docs.oracle.com/javase/7/docs/api/java/util/LinkedList.html#addFirst(E) > On Aug 25, 2016, at 2:45 AM, ramkrishna vasudevan > wrote: > > And reading thro the mail chain as Ted

Re: How to get Last 1000 records from 1 millions records

2016-08-25 Thread ramkrishna vasudevan
And reading thro the mail chain as Ted suggested if you setReversedScan as True and reverse your stop and start Row you can just do a count in your Row filter filter till 10k is reached and then just skip all the other results. In the other logic that I had said you may have to do a sort before

Re: How to get Last 1000 records from 1 millions records

2016-08-25 Thread ramkrishna vasudevan
Hi Manjeet For your first question regarding fetching last 1000 records First in your scan you set your start Row with the bytes corresponding to ( A_98_) and let the end byte be the byte representation of A_98 + 1 . I mean add +1 to the last byte of what comes out of

Re: How to get Last 1000 records from 1 millions records

2016-08-25 Thread Manjeet Singh
Hi All I have one another question for same case below is my sample Hbase data as we all know that hbase store data on the basis of rowkey (sorted) below is IP as we can see 2.168.129.81_1 is in last what I am expecting it shuld come just after 1.168.129.81_2 1.168.129.81_0

Re: How to get Last 1000 records from 1 millions records

2016-08-25 Thread Manjeet Singh
I am using some logical salt say I have mobile number in my row key so I am using some algo and fitting this mobile number into some ASCII char So each time I know what will be the salt so its clear to me and it will never change the order example if based on my algo I get A for 98 so each

Re: How to get Last 1000 records from 1 millions records

2016-08-24 Thread Dima Spivak
Hey Manjeet, How much data are you actually trying to get the last 1000 records for? If you're dealing at the scale of only millions of rows, HBase may not be the best choice for this type of problem. On Wed, Aug 24, 2016 at 12:05 PM, Manjeet Singh wrote: > Hi all >

Re: How to get Last 1000 records from 1 millions records

2016-08-24 Thread Ted Yu
Manjeet didn't mention hashing. Nor did the example show hashed keys. Cheers On Wed, Aug 24, 2016 at 12:36 PM, Esteban Gutierrez wrote: > As long as new rows are added to the latest region that "might" work. But > if the table is using hashed keys or rows are added

Re: How to get Last 1000 records from 1 millions records

2016-08-24 Thread Esteban Gutierrez
As long as new rows are added to the latest region that "might" work. But if the table is using hashed keys or rows are added randomly to the table then retrieving the last million will be trickier and you will have to scan based on timestamp (if not modified) and then filter one more time.

Re: How to get Last 1000 records from 1 millions records

2016-08-24 Thread Ted Yu
The following API should help in your case: public Scan setReversed(boolean reversed) { Cheers On Wed, Aug 24, 2016 at 12:05 PM, Manjeet Singh wrote: > Hi all > > Hbase didnt provide sorting on column but rowkey store in sorted form > like small value first and

How to get Last 1000 records from 1 millions records

2016-08-24 Thread Manjeet Singh
Hi all Hbase didnt provide sorting on column but rowkey store in sorted form like small value first and greater value last example 1 2 3 4 5 6 7 and so on Assume I have 1 Miilions record but i want to look last 1000 records only Is their any way to do this? I don't want to perform any