Re: Using Hbase as data sink

2008-12-26 Thread stack
Jim Twensky wrote: ... I tried to look around to find a method to get the partitioner via JobConf but there is no such thing. Does http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/JobConf.html#setPartitionerClass(java.lang.Class) not work for you? St.Ack

Re: Using Hbase as data sink

2008-12-24 Thread Jim Twensky
Hi again, Pardon me but which 'run' method? Why do you not have access? Its a public class? (Sorry if I'm missing an obvious -- still on first cup of coffee). So here is how my class looks like: public class PhraseGenerator extends Configured implements Tool { ... public int

RE: Using Hbase as data sink

2008-12-22 Thread Jonathan Gray
Jim, This looks like a sane way to do what you want. Is your question strictly on other methods to put the same data layout into HBase from the MR job, or also about the choice of structure? As far as how else to use HBase as a data sink, you can make use of TableOutputFormat. In my

Re: Using Hbase as data sink

2008-12-22 Thread Jim Twensky
Hello Jonathan, Thanks for the fast response. Yes, my question is on other methods to put the same data layout into HBase from my map reduce jobs. I've seen the TableOutputFormat but I couldn't find any example usages of it. Specifically, when we use the FileOutputFormat, a file called part

Re: Using Hbase as data sink

2008-12-22 Thread Jim Twensky
Thank you, that really helped, I appreciate it. I have a final question about the following code you posted: if (partitioner != null) { job.setPartitionerClass(HRegionPartitioner.class); HTable outputTable = new HTable(new HBaseConfiguration(job), table); int regions =