MultiTableInputFormat to compare 2 tables taking about 80 mins

2014-11-03 Thread krish_571
Hi, I am using Hbase MultiTableInputFormat to compare 2 tables: Table1 (7 million), Table2 (30 million). In the driver, i am passing to scans ( without any filters). In my mapper i am doing a compare and writing the summary in Reducer. Any settings specific to this scenario that might speed up

setup method in Mapreduce Program

2014-10-22 Thread krish_571
I understand that setup and cleanup methods of Mapper/Reducer are called for each task, so if you have 20 mappers running, the setup / cleanup will be called for each one. Is there a setup like method to for the entire mapreduce program. Basically i want to open only 1 connectionto a HBase

Hbase memory leak error message when stopping tomcat

2014-07-07 Thread krish_571
We have a java application (on tomcat) that connects to Hbase. We are getting the below errors when we stop tomcat. Any thoughts? SEVERE: The web application [/testapp] appears to have started a thread named [hbase-tablepool-168-thread-1] but has failed to stop it. This is very likely to create

Limiting number of records in Hbase Scan

2014-05-19 Thread krish_571
Is there any java api to limit the number of scanned records after using start and stop rows? Is pagefilter an option? -- View this message in context: http://apache-hbase.679495.n3.nabble.com/Limiting-number-of-records-in-Hbase-Scan-tp4059401.html Sent from the HBase User mailing list archive