[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050368#comment-13050368 ]
Eran Kutner commented on HBASE-3996: ------------------------------------ I've added three new classes: MultiTableInputCollection is a collection of table+scanner pairs that should be used as input to the mapper. MultiTableInputFormatBase and MultiTableInputFormat are closely based on the "non-multi" version with required adaptations. I've also updated TableMapReduceUtil and TableSplit to support these new classes. Usage example: Scan scan1 = new Scan(); scan1.setStartRow(start1); scan1.setStopRow(end1); Scan scan2 = new Scan(); scan2.setStartRow(start2); scan2.setStopRow(end2); MultiTableInputCollection mtic = new MultiTableInputCollection(); mtic.Add(tableName1, scan1); mtic.Add(tableName2, scan2); TableMapReduceUtil.initTableMapperJob(mtic, TestTableMapper.class, Text.class, IntWritable.class, job1); > Support multiple tables and scanners as input to the mapper in map/reduce jobs > ------------------------------------------------------------------------------ > > Key: HBASE-3996 > URL: https://issues.apache.org/jira/browse/HBASE-3996 > Project: HBase > Issue Type: Improvement > Components: mapreduce > Reporter: Eran Kutner > Fix For: 0.90.4 > > Attachments: MultiTableInputCollection.java, > MultiTableInputFormat.java, MultiTableInputFormatBase.java, > TableMapReduceUtil.java, TableSplit.java > > > It seems that in many cases feeding data from multiple tables or multiple > scanners on a single table can save a lot of time when running map/reduce > jobs. > I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira