Comments in-line below:
Naama Kraus wrote:
Here is an updated code
Naama
/**
* <pre>
* 'Toy tables' for experiencing with MapReduce over HBase
Do you mean 'experimenting' in the above?
....
public void create() throws IOException {
Where does this method get called? I don't see it.
System.out.println("Grades Table populated");
Do you want to set up a logger to do the outputting instead? See the
head of (most) hbase classes for example. Look for 'LOG'.
}
}
====================================================
/**
* A map reduce job over [EMAIL PROTECTED] GradesTable}
* The job produces for each course the average grade in that course.
* It puts the average in a separate table which holds course statistics.
*
*/
public class GradesTableMapReduce extends Configured implements Tool {
/**
* Map a row to {key, value} pairs.
* Emit a {course, grade} pair for each course grade appearing in the
student row.
* E.g. Sara {Math:62, Art:45, Sports:87} -> {Math, 62}, {Art, 45},
{Sports, 87}
*
*/
public static class GradesTableMap extends TableMap<Text, IntWritable> {
@Override
public void map(HStoreKey key, MapWritable value,
OutputCollector<Text, IntWritable> output, Reporter reporter) throws
IOException {
// Walk through the columns
for (Map.Entry<Writable, Writable> e: value.entrySet()) {
// Column name is course name
Text course = (Text) e.getKey();
// Remove the family prefix
String courseStr = course.toString();
courseStr =
courseStr.substring(courseStr.indexOf(':') + 1);
There may be utility in HStoreKey to do the above stripping of the
column family (getQualifier?).
course = new Text(courseStr);
byte [] gradeInBytes = ((ImmutableBytesWritable)
e.getValue()).get();
DataInputStream in = new DataInputStream(new
ByteArrayInputStream(gradeInBytes));
IntWritable grade = new IntWritable();
grade.readFields(in);
You could have used Writables.getWritable above and saved yourself a few
lines (Not important).
Otherwise, this class is an excellent example of using MR + HBase. I've
add a pointer to it up on the wiki under the MR+HBase page (update the
link if you update your code).
Thanks,
St.Ack
