[
https://issues.apache.org/jira/browse/KYLIN-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318102#comment-15318102
]
liyang commented on KYLIN-1726:
-------------------------------
Hi Ken, the offset on CubeSegment can be the sum of all partition offsets. I
need an offset concept to logically cut cube into segments, to put segments in
order, and to check no overlap/hole between segments.
There need to be a separate field (or a separate json?) to hold the concrete
offsets for each partition. Also need to think about partition addition etc.
> Scalable streaming cubing
> -------------------------
>
> Key: KYLIN-1726
> URL: https://issues.apache.org/jira/browse/KYLIN-1726
> Project: Kylin
> Issue Type: New Feature
> Reporter: liyang
> Assignee: liyang
>
> We try to achieve:
> 1. Scale streaming cubing workload on a computation cluster, e.g. YARN
> 2. Support Kafka as a formal data source
> 3. Guarantee no data loss reading from Kafka, even records are not strictly
> ordered by time
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
