Question about the time to execute joins in HBase!

2013-08-22 Thread Pavan Sudheendra
Hi all, A serious question.. I know this isn't one of the best hbase practices but I really want to know.. I am doing a join across 3 table in hbase.. One table contain 19m records, one contains 2m and another contains 1m records. I'm doing this inside the mapper function.. I know this can be

Re: Question about the time to execute joins in HBase!

2013-08-22 Thread Kevin O'dell
QQ what is your caching set to? On Aug 22, 2013 11:25 AM, Pavan Sudheendra pavan0...@gmail.com wrote: Hi all, A serious question.. I know this isn't one of the best hbase practices but I really want to know.. I am doing a join across 3 table in hbase.. One table contain 19m records, one

Re: Question about the time to execute joins in HBase!

2013-08-22 Thread Pavan Sudheendra
scan.setCaching(500); I really don't understand this purpose though.. On Thu, Aug 22, 2013 at 9:09 PM, Kevin O'dell kevin.od...@cloudera.comwrote: QQ what is your caching set to? On Aug 22, 2013 11:25 AM, Pavan Sudheendra pavan0...@gmail.com wrote: Hi all, A serious question.. I know

Re: Question about the time to execute joins in HBase!

2013-08-22 Thread Jean-Marc Spaggiari
And size of the rows... can you load the 1m rows table in memory? Le 2013-08-22 11:41, Pavan Sudheendra pavan0...@gmail.com a écrit : scan.setCaching(500); I really don't understand this purpose though.. On Thu, Aug 22, 2013 at 9:09 PM, Kevin O'dell kevin.od...@cloudera.com wrote: QQ

Re: Question about the time to execute joins in HBase!

2013-08-22 Thread Pavan Sudheendra
Hmmm. I'm not sure about this.. How do i check Jean? On Thu, Aug 22, 2013 at 9:12 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: And size of the rows... can you load the 1m rows table in memory? Le 2013-08-22 11:41, Pavan Sudheendra pavan0...@gmail.com a écrit :

Re: Question about the time to execute joins in HBase!

2013-08-22 Thread Pavan Sudheendra
Yes Michael i think so.. I was googling about what you said.. I'm afraid i'm not aware of most of the terms.. I'm still yet to learn but don't have much time. :( On Thu, Aug 22, 2013 at 9:16 PM, Michael Segel michael_se...@hotmail.comwrote: You kind of have two threads along the same lines.

Re: Question about the time to execute joins in HBase!

2013-08-22 Thread Michael Segel
You kind of have two threads along the same lines. See my response in your other thread... On Aug 22, 2013, at 10:41 AM, Pavan Sudheendra pavan0...@gmail.com wrote: scan.setCaching(500); I really don't understand this purpose though.. On Thu, Aug 22, 2013 at 9:09 PM, Kevin O'dell

Re: Question about the time to execute joins in HBase!

2013-08-22 Thread Michael Segel
Pig and Hive will generate a map/reduce job So you have 3 tables that you want to join. Ok so one is 60 million rows. One is 2 million and 1 is 1 million. What sort of join? Can you write your join in terms of a relationship? Could you write it a SQL like code? Join table A to table B ON