[
https://issues.apache.org/jira/browse/KYLIN-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928306#comment-17928306
]
Guoliang Sun commented on KYLIN-6028:
-------------------------------------
h3. Dev Design
There are two scenarios where `dataflow` and `segments` may become
inconsistent:
1. The segment UUIDs recorded in `dataflow` are fewer than the segments in the
cache:
No action is required. When `dataflow` retrieves the segment list, it always
fetches based on the UUIDs list. Stale segment caches will be ignored.
2. The segment UUIDs recorded in `dataflow` are more than the segments in the
cache:
This situation can occur during concurrent build/refresh operations. Since
adding a new segment requires checking for overlaps or duplicate refreshes with
other segments, it is necessary to obtain the latest segment list. In this
case, a forced metadata catch-up can be attempted to ensure consistency.
> Kylin5 encounters metadata anomalies when concurrently submitting
> build/refresh tasks
> -------------------------------------------------------------------------------------
>
> Key: KYLIN-6028
> URL: https://issues.apache.org/jira/browse/KYLIN-6028
> Project: Kylin
> Issue Type: Bug
> Affects Versions: 5.0.0
> Reporter: Guoliang Sun
> Priority: Major
>
> In Kylin5, when two incremental build tasks with the same time range for the
> same model are submitted concurrently, both requests succeed. However, only
> one segment is created for the model, while two build tasks are created,
> which is inconsistent with expectations.
> Further verification shows that the same issue occurs when concurrently
> refreshing the same segment.
> Additional testing reveals that submitting build/refresh tasks concurrently
> for a model may result in issues, regardless of whether these tasks conflict
> or not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
