[
https://issues.apache.org/jira/browse/KYLIN-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716495#comment-17716495
]
Yaguang Jia commented on KYLIN-5527:
------------------------------------
Dev Design is attached, both in English and Chinese
> Kylin job engine adds the ability to check entries with data source(e.g. hive)
> ------------------------------------------------------------------------------
>
> Key: KYLIN-5527
> URL: https://issues.apache.org/jira/browse/KYLIN-5527
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
> Affects Versions: 5.0-alpha
> Reporter: Yaguang Jia
> Assignee: Yaguang Jia
> Priority: Critical
> Fix For: 5.0-beta
>
> Attachments: (Chinese)KYLIN-5527 Kylin job engine adds the ability to
> check entries with data source(e.g. hive).pdf, (English) KYLIN-5527 Kylin job
> engine adds the ability to check entries with data source(e.g. hive).pdf
>
>
> After adding some new indexes in the Kylin5 model, the operation of
> completing the indexes is needed. If the previous data in the data source
> (e.g.:hive) has been deleted, then there will be a situation that the newly
> added index data is empty, which will lead to wrong query results. Here we
> need to add a new data source check function, if the data in the data source
> has been deleted, then do not fill in the data of the corresponding time
> segment to ensure the correctness of the query results.
>
> 在Kylin5的模型中新增加一些索引后，需要进行补齐索引的操作，如果数据源（如:hive）里之前的数据被删掉了，那么就会出现新补出来的索引数据为空的情况，从而导致查询结果有误。这里需要新增加一个数据源核对的功能，如果数据源中的数据已经被删掉了，那么就不去补对应时间segment的数据，保证查询结果的正确性。
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
