Re: write data into HBase via spark

2013-12-06 Thread Philip Ogren
Hao, Thank you for the detailed response! (even if delayed!) I'm curious to know what version of hbase you added to your pom file. Thanks, Philip On 11/14/2013 10:38 AM, Hao REN wrote: Hi, Philip. Basically, we need* PairRDDFunctions.saveAsHadoopDataset* to do the job, as HBase is not a

RE: write data into HBase via spark

2013-12-06 Thread Benjamin Kim
: Fri, 6 Dec 2013 18:07:08 -0700 From: philip.og...@oracle.com To: user@spark.incubator.apache.org Subject: Re: write data into HBase via spark Hao, Thank you for the detailed response! (even if delayed!) I'm curious to know what version of hbase you added

Re: write data into HBase via spark

2013-11-14 Thread Hao REN
Hi, Philip. Basically, we need* PairRDDFunctions.saveAsHadoopDataset* to do the job, as HBase is not a fs, saveAsHadoopFile doesn't work. *def saveAsHadoopDataset(conf: JobConf): Unit* this function takes a JobConf parameter which should be configured. Essentially, you need to set output format

Re: write data into HBase via spark

2013-11-13 Thread Hao REN
Ok, I worked it out. The following thread helps a lot. http://mail-archives.apache.org/mod_mbox/incubator-spark-user/201310.mbox/%3C7B4868A9-B83E-4507-BB2A-2721FCE8E738%40gmail.com%3E Hao 2013/11/12 Hao REN julien19890...@gmail.com Could someone show me a simple example about how to write

Re: write data into HBase via spark

2013-11-13 Thread Philip Ogren
Hao, If you have worked out the code and turn it into an example that you can share, then please do! This task is in my queue of things to do so any helpful details that you uncovered would be most appreciated. Thanks, Philip On 11/13/2013 5:30 AM, Hao REN wrote: Ok, I worked it out.