Re: S3 as kylin storage.

[ShaoFeng Shi]

Agree with Yaqian; S3 is object storage, while HBase is a fast K-V storage.
S3 is good for big objects like files, video, images, etc, but not suitable
for small objects like a number, a string, etc. You can use S3 as the DFS
for Kylin (like HDFS), and also can use S3 as HBase's underlying storage.
These are all supported, many AWS users are running Kylin on HBase+S3.

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofeng...@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscr...@kylin.apache.org
Join Kylin dev mail group: dev-subscr...@kylin.apache.org




Yaqian Zhang  于2019年11月21日周四 上午10:43写道：

> Hi:
>
> I think your understanding is correct，S3 can’t instead of Hbase.
>
> You should have read this article.
>
> http://kylin.apache.org/docs/install/kylin_aws_emr.html <
> http://kylin.apache.org/docs/install/kylin_aws_emr.html>.
>
> This introduce how to use S3 as kylin.env.hdfs-working-dir, andassume
> HBase is also on S3.
>
> > 在 2019年11月19日，20:11，Tanmay Movva  写道：
> >
> > Hello,
> >
> > We have decided to use kylin to build our data warehouse and are going to
> > setup kylin on a cluster soon. We would like to know if S3 can be used
> as a
> > storage for apache kylin instead of Hbase as both are key-value stores
> and
> > query directly on S3.
> >
> > From what I understand I can only use hbase.storage_mode = s3 but not S3
> as
> > a direct storage for kylin cubes. Please help me out here, Thanks.
> >
> > --
> > Regards,
> > Tanmay Krishna Movva
> > Razorpay
>
>

Re: S3 as kylin storage.

[Yaqian Zhang]

Hi:

I think your understanding is correct，S3 can’t instead of Hbase.

You should have read this article.

http://kylin.apache.org/docs/install/kylin_aws_emr.html 
.

This introduce how to use S3 as kylin.env.hdfs-working-dir, andassume HBase is 
also on S3.

> 在 2019年11月19日，20:11，Tanmay Movva  写道：
> 
> Hello,
> 
> We have decided to use kylin to build our data warehouse and are going to
> setup kylin on a cluster soon. We would like to know if S3 can be used as a
> storage for apache kylin instead of Hbase as both are key-value stores and
> query directly on S3.
> 
> From what I understand I can only use hbase.storage_mode = s3 but not S3 as
> a direct storage for kylin cubes. Please help me out here, Thanks.
> 
> -- 
> Regards,
> Tanmay Krishna Movva
> Razorpay

S3 as kylin storage.

[Tanmay Movva]

Hello,

We have decided to use kylin to build our data warehouse and are going to
setup kylin on a cluster soon. We would like to know if S3 can be used as a
storage for apache kylin instead of Hbase as both are key-value stores and
query directly on S3.

>From what I understand I can only use hbase.storage_mode = s3 but not S3 as
a direct storage for kylin cubes. Please help me out here, Thanks.

-- 
Regards,
Tanmay Krishna Movva
Razorpay