Re: 答复: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-13 Thread Ted Malaska
, and use Gets as much as possible. Thanks, *发件人:* Ted Malaska [mailto:ted.mala...@cloudera.com] *发送时间:* 2015年8月12日 9:14 *收件人:* Yan Zhou.sc *抄送:* dev@spark.apache.org; Bing Xiao (Bing); Ted Yu; user *主题:* RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro There a number

RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Ted Malaska
, August 12, 2015 7:28 AM *To:* Yan Zhou.sc *Cc:* user; dev@spark.apache.org; Bing Xiao (Bing); Ted Yu *Subject:* RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro Hey Yan, I've been the one building out this spark functionality in hbase so maybe I can help clarify

答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Yan Zhou.sc
@spark.apache.orgmailto:dev@spark.apache.org; u...@spark.apache.orgmailto:u...@spark.apache.org; Yan Zhou.sc 主题: Re: Package Release Annoucement: Spark SQL on HBase Astro Yan / Bing: Mind taking a look at HBASE-14181https://issues.apache.org/jira/browse/HBASE-14181 'Add Spark DataFrame DataSource to HBase-Spark

Re: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Ted Yu
, *From:* Ted Yu [mailto:yuzhih...@gmail.com] *Sent:* Tuesday, August 11, 2015 3:28 PM *To:* Yan Zhou.sc *Cc:* Bing Xiao (Bing); dev@spark.apache.org; u...@spark.apache.org *Subject:* Re: 答复: Package Release Annoucement: Spark SQL on HBase Astro HBase will not have query engine

答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Yan Zhou.sc
: 2015年8月11日 8:54 收件人: Bing Xiao (Bing) 抄送: dev@spark.apache.org; u...@spark.apache.org; Yan Zhou.sc 主题: Re: Package Release Annoucement: Spark SQL on HBase Astro Yan / Bing: Mind taking a look at HBASE-14181https://issues.apache.org/jira/browse/HBASE-14181 'Add Spark DataFrame DataSource

答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Yan Zhou.sc
[mailto:yuzhih...@gmail.com] 发送时间: 2015年8月11日 15:28 收件人: Yan Zhou.sc 抄送: Bing Xiao (Bing); dev@spark.apache.org; u...@spark.apache.org 主题: Re: 答复: Package Release Annoucement: Spark SQL on HBase Astro HBase will not have query engine. It will provide better support to query engines. Cheers On Aug 10, 2015

Re: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Ted Yu
, …, etc., which allows for loosely-coupled query engines built on top of it. Thanks, 发件人: Ted Yu [mailto:yuzhih...@gmail.com] 发送时间: 2015年8月11日 8:54 收件人: Bing Xiao (Bing) 抄送: dev@spark.apache.org; u...@spark.apache.org; Yan Zhou.sc 主题: Re: Package Release Annoucement: Spark SQL

RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Ted Malaska
[mailto:ted.mala...@cloudera.com] *Sent:* Wednesday, August 12, 2015 7:28 AM *To:* Yan Zhou.sc *Cc:* user; dev@spark.apache.org; Bing Xiao (Bing); Ted Yu *Subject:* RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro Hey Yan, I've been the one building out this spark functionality

答复: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Yan Zhou.sc
); Ted Yu; user 主题: RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro There a number of ways to bulk load. There is bulk put, partition bulk put, mr bulk load, and now hbase-14150 which is spark shuffle bulk load. Let me know if I have missed a bulk loading option. All these r

RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Yan Zhou.sc
To: Yan Zhou.sc Cc: user; dev@spark.apache.org; Bing Xiao (Bing); Ted Yu Subject: RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro Hey Yan, I've been the one building out this spark functionality in hbase so maybe I can help clarify. The hbase-spark module is just focused

RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Yan Zhou.sc
@spark.apache.org; Ted Yu; Bing Xiao (Bing); user Subject: RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro The bulk load code is 14150 if u r interested. Let me know how it can be made faster. It's just a spark shuffle and writing hfiles. Unless astro wrote it's own shuffle

RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Ted Malaska
as integers from Astro. Thanks, *From:* Ted Yu [mailto:yuzhih...@gmail.com] *Sent:* Wednesday, August 12, 2015 7:02 AM *To:* Yan Zhou.sc *Cc:* Bing Xiao (Bing); dev@spark.apache.org; u...@spark.apache.org *Subject:* Re: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro Yan

RE: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-11 Thread Yan Zhou.sc
Subject: Re: 答复: 答复: Package Release Annoucement: Spark SQL on HBase Astro Yan: Where can I find performance numbers for Astro (it's close to middle of August) ? Cheers On Tue, Aug 11, 2015 at 3:58 PM, Yan Zhou.sc yan.zhou...@huawei.commailto:yan.zhou...@huawei.com wrote: Finally I can take a look

Re: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-10 Thread Ted Yu
Yan / Bing: Mind taking a look at HBASE-14181 https://issues.apache.org/jira/browse/HBASE-14181 'Add Spark DataFrame DataSource to HBase-Spark Module' ? Thanks On Wed, Jul 22, 2015 at 4:53 PM, Bing Xiao (Bing) bing.x...@huawei.com wrote: We are happy to announce the availability of the Spark

Re: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-03 Thread Ted Yu
When I tried to compile against hbase 1.1.1, I got: [ERROR] /home/hbase/ssoh/src/main/scala/org/apache/spark/sql/hbase/SparkSqlRegionObserver.scala:124: overloaded method next needs result type [ERROR] override def next(result: java.util.List[Cell], limit: Int) = next(result) Is there plan to

RE: Package Release Annoucement: Spark SQL on HBase Astro

2015-08-03 Thread Yan Zhou.sc
...@spark.apache.org; Yan Zhou.sc Subject: Re: Package Release Annoucement: Spark SQL on HBase Astro When I tried to compile against hbase 1.1.1, I got: [ERROR] /home/hbase/ssoh/src/main/scala/org/apache/spark/sql/hbase/SparkSqlRegionObserver.scala:124: overloaded method next needs result type [ERROR

Re: Package Release Annoucement: Spark SQL on HBase Astro

2015-07-28 Thread Debasish Das
*Cc:* Bing Xiao (Bing); dev; user *Subject:* RE: Package Release Annoucement: Spark SQL on HBase Astro Hi Yan, Is it possible to access the hbase table through spark sql jdbc layer ? Thanks. Deb On Jul 22, 2015 9:03 PM, Yan Zhou.sc yan.zhou...@huawei.com wrote: Yes, but not all SQL

RE: Package Release Annoucement: Spark SQL on HBase Astro

2015-07-27 Thread Debasish Das
, 2015 7:36 PM *To:* Bing Xiao (Bing) *Cc:* user; dev; Yan Zhou.sc *Subject:* Re: Package Release Annoucement: Spark SQL on HBase Astro Does it also support insert operations ? On Jul 22, 2015 4:53 PM, Bing Xiao (Bing) bing.x...@huawei.com wrote: We are happy to announce the availability

RE: Package Release Annoucement: Spark SQL on HBase Astro

2015-07-27 Thread Yan Zhou.sc
- HBase Scans/Gets - … - HBase Region server Regards, Yan From: Debasish Das [mailto:debasish.da...@gmail.com] Sent: Monday, July 27, 2015 10:02 PM To: Yan Zhou.sc Cc: Bing Xiao (Bing); dev; user Subject: RE: Package Release Annoucement: Spark SQL on HBase Astro Hi Yan, Is it possible to access

RE: Package Release Annoucement: Spark SQL on HBase Astro

2015-07-22 Thread Yan Zhou.sc
Yes, but not all SQL-standard insert variants . From: Debasish Das [mailto:debasish.da...@gmail.com] Sent: Wednesday, July 22, 2015 7:36 PM To: Bing Xiao (Bing) Cc: user; dev; Yan Zhou.sc Subject: Re: Package Release Annoucement: Spark SQL on HBase Astro Does it also support insert operations

Re: Package Release Annoucement: Spark SQL on HBase Astro

2015-07-22 Thread Debasish Das
Does it also support insert operations ? On Jul 22, 2015 4:53 PM, Bing Xiao (Bing) bing.x...@huawei.com wrote: We are happy to announce the availability of the Spark SQL on HBase 1.0.0 release. http://spark-packages.org/package/Huawei-Spark/Spark-SQL-on-HBase The main features in this

Package Release Annoucement: Spark SQL on HBase Astro

2015-07-22 Thread Bing Xiao (Bing)
We are happy to announce the availability of the Spark SQL on HBase 1.0.0 release. http://spark-packages.org/package/Huawei-Spark/Spark-SQL-on-HBase The main features in this package, dubbed Astro, include: * Systematic and powerful handling of data pruning and intelligent scan, based