[
https://issues.apache.org/jira/browse/KYLIN-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16820980#comment-16820980
]
XiaoXiang Yu commented on KYLIN-3962:
-------------------------------------
If I use Flink streaming to ingest streaming message, and write cuboid data(I
think this maybe <Dimension Array, MeasureAggregator>) to external storage(like
hbase or redis), I think it will have some drawback:
- Using remote storage instead of local storage is will increase the data
prepare delay.
- Introduce external dependency like flink cluster.
- Too heavy pressure on external storage.
- Filter and aggregate will be slower if data in remote.
> Support streaming cubing using Spark Streaming or Flink
> -------------------------------------------------------
>
> Key: KYLIN-3962
> URL: https://issues.apache.org/jira/browse/KYLIN-3962
> Project: Kylin
> Issue Type: Improvement
> Reporter: Liu Shaohui
> Priority: Major
>
> KYLIN-3654 introduced the Real-time Streaming, but in my opinion, the arch is
> a little too complicated to handle.
> As streaming frameworks like spark streaming, flink are widely used in many
> companies.Can we use the streaming framework to support real time cubing in
> Kylin.
> This is just a proposal. More discussion and suggestions are welcomed~
> More details of this proposal will be added later.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
